Unloading NativeAOT compiled dll

Question

With .NET 7's NativeAOT compilation. We can now load a C# dll as regular Win32 module.

HMODULE module = LoadLibraryW("AOT.dll");
auto hello = GetProcAddress(module, "Hello");
hello();

This works fine and prints some stuff in console.

However, when unloading the dll. It simply doesn't work. No matter how many times I call FreeLibrary("AOT.dll"), GetModuleHandle("AOT.dll") still returns the handle to the module, implying that it did not unload successfully.

My "wild guess" was that the runtime has some background threads still running (GC?), so I enumerated all threads and use NtQueryInformationThread to retrive the start address of each thread then call GetModuleHandleEx with GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS to get the module where the thread started, the result were as follows.

Before:

THREAD ID      = 7052
base priority  = 8
delta priority = 0
Start address: 00007FF69D751613
Module: 00007FF69D740000 => CppRun.exe


THREAD ID      = 3248
base priority  = 8
delta priority = 0
Start address: 00007FFEF1F42B20
Module: 00007FFEF1EF0000 => ntdll.dll


THREAD ID      = 7160
base priority  = 8
delta priority = 0
Start address: 00007FFEF1F42B20
Module: 00007FFEF1EF0000 => ntdll.dll

After:

THREAD ID      = 7052
base priority  = 8
delta priority = 0
Start address: 00007FF69D751613
Module: 00007FF69D740000 => CppRun.exe


THREAD ID      = 3248
base priority  = 8
delta priority = 0
Start address: 00007FFEF1F42B20
Module: 00007FFEF1EF0000 => ntdll.dll


THREAD ID      = 7160
base priority  = 8
delta priority = 0
Start address: 00007FFEF1F42B20
Module: 00007FFEF1EF0000 => ntdll.dll


THREAD ID      = 5944
base priority  = 8
delta priority = 0
Start address: 00007FFEF1F42B20
Module: 00007FFEF1EF0000 => ntdll.dll


THREAD ID      = 17444
base priority  = 10
delta priority = 0
Start address: 00007FFE206DBEF0
Module: 00007FFE206D0000 => AOT.dll

"CppRun.exe" is my testing application.

As you can see, two additional threads were spawned. One from ntdll (5944), and one from my AOT compiled dll (17444).

I don't know what the leftover thread in "AOT.dll" was for (maybe GC?), but I force-terminated it successfully (definitely unhealthy, I know).

However, when I tried to open the thread in ntdll (5944), it throws an exception

An invalid thread, handle %p, is specified for this operation. Possibly, a threadpool worker thread was specified

Given that, I assume .NET starts a threadpool worker during initilization? How can I stop that pool and unload the dll?

Or, is there a better way for unloading a NativeAOT compiled dll?

Update: I've hooked the CreateThreadPool function, but the runtime doesn't call it. Still trying to figure out what spawned that thread.

Answer 1

Edit:

NativeAOT(aka CoreRT) compiled dll was unloadable at first, but Microsoft later blocked the functionality due to memory leak and crash on process exit. See this PR for more details. This answer simply restores the functionality using detour hook and does not deal with the memory leak nor the crash. Use it at your own risk.

I was able to prevent the access violation crash by manually freeing the FLS(fiber-local storage) created by .NET. Here is a simple demo.

Original answer below:

Turns out that thread is used by Windows 10 for parallel library loading(TppWorkerThread) and isn't the problem.

I ended up inspecting the winapi call with this handy tool, and found that .NET is calling GetModuleHandleEx with the GET_MODULE_HANDLE_EX_FLAG_PIN flag, thus preventing the module from unloading.

So I hooked GetModuleHandleEx to intercept calls and shift out the flag. Voila! Now I can unload the NativeAOT compiled dll.

I know this approach is quite hacky, but at least it works. If anyone happen to have a better solution, please let me know.