1

I'm trying to write a c-extension for python and I'm using a library (FFTW) which has some dynamic library files (.dll). So I wrote some c code and also a setup.py file to install it and when I run the setup file everything goes well and my c-extensions will be installed successfully. But when I try to use my new module I will encounter the following error:

ImportError: DLL load failed: The specified module could not be found.

So I assume it is referring to the dll files of FFTW library. If I copy the dll files to my C:/windows/system32 My code will run without any error. But I wish not to do that and instead have the dll files alongside my code. According to python documentation for writing the setup script:

You can also specify the libraries to link against when building your extension, and the directories to search for those libraries. The libraries option is a list of libraries to link against, library_dirs is a list of directories to search for libraries at link-time, and runtime_library_dirs is a list of directories to search for shared (dynamically loaded) libraries at run-time.

But when I use the runtime_library_dirs option and try to install my extension by running the setup.py I will encounter the following error.

warning: I don't know what to do with 'runtime_library_dirs': ['./c-extension/bin'] error: don't know how to set runtime library search path for MSVC

I'm using python 3.6.

Here is a sample of my setup.py code:

import numpy
from distutils.core import setup, Extension

def main():
    setup(name="simple_ext",
          version="1.0.0",
          description="Python interface for the simple C extension library function",
          author="My Name",
          author_email="my_email@email.com",
          data_files=[('./c-extension/bin', ['libfftw3-3.dll','libfftw3l-3.dll','libfftw3f-3.dll'])],
          ext_modules=[Extension("simple_ext", ["simple_ext.cpp"],
                  include_dirs=[numpy.get_include(), './c-extension/include']),
                  runtime_library_dirs=['./c-extension/bin'],
                  library_dirs=['./c-extension/lib'],
                  libraries = ['libfftw3-3','libfftw3l-3','libfftw3f-3']])

if __name__ == "__main__":
    main()

So does anyone any idea what wrong and what should I do to fix it?

Edit:

So based on Patrik Polakovic's comment I managed to somehow solve the problem. The thing is when I used data_files like this:

data_files=[('./c-extension/bin', ['libfftw3-3.dll','libfftw3l-3.dll','libfftw3f-3.dll'])],

There was an hidden error message that I had missed and apparently was ignored by the compiler too.

error: can't copy 'libfftw3-3.dll': doesn't exist or not a regular file

But when I changed it to the form below:

data_files=[('', ['./c-extension/bin/libfftw3-3.dll','./c-extension/bin/libfftw3l-3.dll','./c-extension/bin/libfftw3f-3.dll'])],

the setup was able to locate my shared library files and the following message was also displayed:

copying c-extension\bin\libfftw3-3.dll -> C:\Users\user\.conda\envs\newEnv\

copying c-extension\bin\libfftw3l-3.dll -> C:\Users\user\.conda\envs\newEnv\

copying c-extension\bin\libfftw3f-3.dll -> C:\Users\user\.conda\envs\newEnv\

So I can use this as a temporary solution. But I still don't know what's wrong with the first form of data_files option that I used (Which looks legit to me) and more importantly I don't understand what's wrong with using the runtime_library_dirs option.

whoAmI
  • 358
  • 4
  • 16
  • 1
    I used *data_files* argument to my *setup* method. `data_files=[("lib/site-packages/mypkg", [get_dll_path()])],`. I don't know if this is correct approach but it did the job. – Patrik Polakovic Jul 27 '20 at 10:25
  • @PatrikPolakovic Thanks for sharing this info with me. I somehow managed to solve the problem using the data_files argument. But I'm still not sure what's wrong with using the runtime_library_dirs argument. I updated the question. – whoAmI Jul 28 '20 at 05:29
  • 2
    very similar: https://stackoverflow.com/q/62662816/5769463 – ead Jul 28 '20 at 12:46
  • 1
    The problem with `runtime_library_dirs` is that it translates to `rpath`, which exists only on Linux but not Windows. – ead Jul 28 '20 at 12:49
  • Thanks for the link @ead. And you are right `runtime_library_dirs` does work on Linux. – whoAmI Jul 29 '20 at 05:08

1 Answers1

2

Yes, on Windows runtime_library_dirs does not work and will cause MSVC to error.

In your original question, you used data_files incorrectly, as from the relevant part of the setuptools docs on writing the setup script, note the sentence

Each (directory, files) pair in the sequence specifies the installation directory and the files to install there.

Your DLLs are located in ./c-extension/bin, so you need to prepend that relative path to the names of each DLL. The following from the same page explains directory concisely:

The directory should be a relative path. It is interpreted relative to the installation prefix (Python’s sys.prefix for system installations; site.USER_BASE for user installations).

That is, the files specified in files will be copied to C:\Users\user\.conda\envs\newEnv\directory. So passing ./c-extension/bin to directory is also a mistake.

phetdam
  • 96
  • 1
  • 8