Load Python C Extension DLL using System.loadLibrary

Question

I have a Python C Extension DLL with extension pyd, that I would like to load in Java through System.loadLibrary on windows. It seems that there is no way to load a dll with an extension different from *.dll.

Is there any suggested approach to overcome this issue?

Use Case

I am planning to embed Python in Java. I created an extension dll which I was loading in JAVA, using JNI to communicate with the extension dll which in turn interacts with the Python Runtime Environment to execute the Python Statements

_____                     _____          _____
| J | System.loadLibrary  |   |          | P |
| A |-------------------->| P |          | Y |
| V |                     | Y |<-------->| T |
| A |       JNI           | D |          | H |
|___|<------------------->|___|          | O |
                                         | N |
                                         |___|

The only thing I can think of would be to either rename the file so it has a `.dll` extension, or create a copy with a `.dll` extension. However, it seems a bit strange to load a `.pyd` in Java. What's the use-case? — Aya, Apr 29 '13 at 18:01

Markku K. · Accepted Answer · 2013-04-29T23:10:47.207

4

Have you tried using System.load() instead? You can specify whatever file name you want, but the downside is that you need the complete path to the .pyd file.

It is not hard to search the paths that a .dll would be loaded from:

for (String s: System.getProperty("java.library.path").split(";")) {
    String pydName = s + "/mypythonlib.pyd";
    File pydFile = new File(pydName);
    if (pydFile.exists()) {
        System.load(pydName);
        break;
    }
}

edited Apr 29 '13 at 23:10

answered Apr 29 '13 at 18:14

Markku K.

3,840
19
20

1

`downside is that you need the complete path to the .pyd file.`, so I didn't try – Abhijit Apr 29 '13 at 18:16
@DavidHeffernan: Can you please explain how it answers the question? – Abhijit Apr 29 '13 at 19:12
1

Well, how about you explain why it fails to answer the question. Anyway, if you want to embed Python, why don't you do that. All the functions you need are found in `pythonXX.dll`. I don't see why you would load a .pyd in Java. What can Java do with it? A .pyd file is for consumption by the Python runtime. – David Heffernan Apr 29 '13 at 19:22

Aya · Answer 2 · 2013-04-29T18:41:20.167

1

Well, if you just want to run Python code in Java, then Jython will let you compile Python source code into Java .class bytecode, but you won't be able to use any CPython functionality which uses extension modules.

If you really need to embed CPython in Java, then 'importing' a .pyd file isn't going to help you. A .pyd will only function correctly if the Python interpreter has already been initialized will a call to Py_Initialize().

What you need to do is look at the CPython embedding docs, and use JNI to call the underlying CPython functions as per the examples in those docs.

Update

It sounds like what you have is a single .dll file which acts as an extension module, but also exposes some symbols you're using to embed Python in Java. If you never plan to import it in CPython, then just rename it to .dll.

If you do need to import it in CPython, then you probably ought to split the functionality into two separate .dll files - one which you use in the JNI embedding, and the other which just acts as an extension module.

edited Apr 29 '13 at 18:41

answered Apr 29 '13 at 18:17

Aya

39,884
6
55
55

Jython unfortunately has lots of limitation esp when trying to use ctypes which would in-turn have issues importing lot of libraries which uses ctypes. And secondly the linked document is very generic and part of it I did undertook to make the interface between C and Python work. Moreover as you mentioned in the comment, when I rename `pyd` to `dll` it works like charm. So the only part which is creating trouble is the one mentioned in the question. – Abhijit Apr 29 '13 at 18:25
@Abhijit It almost sounds like your `.pyd` file is actually exposing more methods than is required for it to function as a `.pyd`. Does it even expose the symbols necessary for it to be imported in CPython? – Aya Apr 29 '13 at 18:31
I am currently not planning to import in Python directly but theoretically I can, bit then again `Does it even expose the symbols necessary for it to be imported in CPython`, how would it impact my design decision? Do you wan't to infer that I shouldn't make it pyd in the first instance? Candidly speaking, I am too contemplating this, but that would create another issue, esp when building pyd through distutil which by default generates a pyd rather than dll. – Abhijit Apr 29 '13 at 18:34
@Abhijit See updated answer, but it's getting to the point where I'm having to speculate too much as to the contents of the `.pyd` file, so it might help to include some of the source code, most notably for the symbols which are `dllexport`ed. – Aya Apr 29 '13 at 18:44

score 1 · Answer 3 · edited May 23 '17 at 12:04

1

See Calling Python in Java?. Basically, you use either org.python.util.PythonInterpreter to call Python or Jython. Another possible solution is to use a UNIX-style .so file. Yet another choice may be to use get the current path, merge it with the name of the .pyd file, and use System.load() as specified by Markku K.

edited May 23 '17 at 12:04

Community

1
1

answered Apr 29 '13 at 18:18

kirbyfan64sos

10,377
6
54
75

Please refer the comment to @Aya's answer to understand why Jython would not work for me. – Abhijit Apr 29 '13 at 18:27
@Abhijit: Your best option is probably to use `System.load()` like this: 1.Get the current directory(`System.getProperty("user.dir")`). 2.Combine the current directory with the name of the library(`full_path = current_dir + name_of_module`). 3.Load the full path using `System.load()`. – kirbyfan64sos Apr 29 '13 at 19:41

Load Python C Extension DLL using System.loadLibrary

3 Answers3