Cannot delete library loaded with JNA after disposing

Question

I am trying to delete a DLL which has been loaded into JNA and later disposed. I have tried all the solutions described in the answer to this question, but they are not working: How to dispose library loaded with JNA

Here is code I've tried without a time delay:

import java.io.File;

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.NativeLibrary;

class Filter {

    private static ExtDLLTool DLLUtil;
    final private static String dllPath = "./ExternalDownloader_64.dll"; 

    static {
        DLLUtil = (ExtDLLTool) Native.loadLibrary(dllPath, ExtDLLTool.class);
    }

    public static void main(String[] args) {

        if (DLLUtil != null) {
            DLLUtil = null;
            NativeLibrary lib = NativeLibrary.getInstance(dllPath);
            lib.dispose();
        }

        File dllFile = new File(dllPath);
        if(dllFile.exists()){
            boolean isDeleted = dllFile.delete();
            if(!isDeleted){
                System.out.println("Unable to delete dll file, since it hold by jvm");
            }
        }

    }

    private interface ExtDLLTool extends Library {
        String validateNomination(String dloadProps);
    }
}

I added a time delay to give the native code time to release the handle:

import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.NativeLibrary;

class Filter {

    private static ExtDLLTool DLLUtil;
    final private static String dllPath = "./ExternalDownloader_64.dll";

    static {
        DLLUtil = (ExtDLLTool) Native.loadLibrary(dllPath, ExtDLLTool.class);
    }

    public static void main(String[] args) throws Exception{

        if (DLLUtil != null) {
            DLLUtil = null;
            NativeLibrary lib = NativeLibrary.getInstance(dllPath);
            lib.dispose();
            Thread.sleep(3000);
        }

        File dllFile = new File(dllPath);
        if(dllFile.exists()){
            Files.delete(Paths.get(dllPath));
            // boolean isDeleted = dllFile.delete();
            if(dllFile.exists()){
                System.out.println("Unable to delete dll file, since it hold by jvm");
            }
        }
    }

    private interface ExtDLLTool extends Library {
        String validateNomination(String dloadProps);
    }
}

This code results in an exception implying the JVM has not released the file.

Exception in thread "main" java.nio.file.AccessDeniedException: .\ExternalDownloader_64.dll at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)

Answer 1

In the end the problem is, that Native#open is called twice and Native#close only once. The assumption behind the presented code is, that:

    NativeLibrary lib = NativeLibrary.getInstance(dllPath);

yields the same NativeLibrary instance, that is used by:

    DLLUtil = (ExtDLLTool) Native.loadLibrary(dllPath, ExtDLLTool.class);

This assumption does not hold. Indeed NativeLibrary#load does use caching and if invoked with the same parameters it will yield only a single instance.

The codepath behind Native.loadLibrary passes two options to Native#loadLibrary: calling-convention and classloader. The calling-convention is equal to the default calling convention, so can be ignored. It is/would be automatically added in NativeLibrary#getInstance. The classloader though is not set to a default value and there is the difference. The options are part of the caching key and thus a second instance of the NativeLibrary is created and not the first returned.

To make it work, the call to NativeLibrary#getInstance must pass the correct classloader. If you modify the sample like this:

import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.NativeLibrary;

class Filter {

    private static ExtDLLTool DLLUtil;
    final private static String dllPath = "./ExternalDownloader_64.dll";

    static {
        DLLUtil = (ExtDLLTool) Native.loadLibrary(dllPath, ExtDLLTool.class);
    }

    public static void main(String[] args) throws Exception{

        if (DLLUtil != null) {
            DLLUtil = null;
            NativeLibrary lib = NativeLibrary.getInstance(dllPath, ExtDLLTool.class.getClassLoader());
            lib.dispose();
            Thread.sleep(3000);
        }

        File dllFile = new File(dllPath);
        if(dllFile.exists()){
            Files.delete(Paths.get(dllPath));
            // boolean isDeleted = dllFile.delete();
            if(dllFile.exists()){
                System.out.println("Unable to delete dll file, since it hold by jvm");
            }
        }
    }

    private interface ExtDLLTool extends Library {
        String validateNomination(String dloadProps);
    }
}

it works as expected.

After discussion there is another requirement: The cache path is only hit in a limited number of cases:

• the library name is the filename of the library (without a prefix)
• the library name is the absolute path to the library
• the library name is the "base" name without any prefixes or suffixes the default library search mechanism adds (on windows ".dll" should be stripped, on linux "lib" prefix and ".so" suffix should be stripped) (UNTESTED!)

The TL;DR version: find the absolute path name and use that for interface loading and NativeLibrary loading.

Answer 2

I was able to reproduce the problem with your code, but only on Windows. When reproducible, I was able to successfully delete the file by adding a garbage collection suggestion before the time delay:

if (DLLUtil != null) {
    DLLUtil = null;
    NativeLibrary lib = NativeLibrary.getInstance(dllPath);
    lib.close();
    System.gc();
    System.gc();
    Thread.sleep(3000);
}

When JNA loads a Windows DLL via Native.loadLibrary(), it internally executes the WinAPI LoadLibraryExW function.

Internally the Java instance is stored in a map to be re-used when possible -- however for this to happen, it requires two things to look up the same Java object:

• the DLL Path must be an absolute path
• the options must match. In this case, you would need to pass the classloader as an argument as Matthias Bläsing indicated in his answer:

// if loaded like this:
DLLUtil = (ExtDLLTool) Native.loadLibrary(dllPath, ExtDLLTool.class);
// fetch from cache like this:
NativeLibrary lib = NativeLibrary.getInstance(dllPath, ExtDLLTool.class.getClassLoader());
lib.dispose();

This should allow you to delete the file.

However, in your case, with the relative path, the library is getting unloaded but the old java object isn't getting closed until GC occurs.

The dispose() (or close() as of 5.12) call in JNA eventually calls the Native.close() method which uses the Windows API FreeLibrary function. This unloads the DLL from the Process memory, so the advice on the linked question on how to dispose is still accurate in the case that you want to re-load the library. If you're not reloading the library, using dispose() (5.11-) or close() (5.12+) is optional.

If you must use a relative path, consider this approach using a PhantomReference inspired by this answer to track the deletion:

if (DLLUtil != null) {
    // Unload the DLL from process memory
    // Optional here, as it will be called by a cleaner on GC below
    NativeLibrary lib = NativeLibrary.getInstance(dllPath);
    lib.close();
    System.out.println("Closed.");

    // Remove any internal JVM references to the file
    final ReferenceQueue rq = new ReferenceQueue();
    final PhantomReference phantom = new PhantomReference(DLLUtil, rq);
    DLLUtil = null;

    // Poll until GC removes the reference
    int count = 0;
    while (rq.poll() == null) {
        System.out.println("Waiting...");
        Thread.sleep(1000);
        if (++count > 4) {
            // After 5 seconds prompt for GC!
            System.out.println("Suggesting GC...");
            System.gc();
        }
    }
    System.out.println("Collected.");
}

The DLL was successfully deleted following this sequence. It did take a second GC call to take effect:

Closed.
Waiting...
Waiting...
Waiting...
Waiting...
Waiting...
Suggesting GC...
Waiting...
Suggesting GC...
Collected.
Deleted!