1

With regard to QAs that touched the subject previously

Situation:

  • I have closed source DLL + .cs header, provided by 3rd party (i also have C .h and .lib version of the same)
  • I've verified that having the library 2 times (under different filenames (see https://stackoverflow.com/a/12533663/492624 ) works, but it requires having the header class twice, under different class names and with different Dll filenames for [DllImport] annotations

What I'm looking for is the solution, that would allow me to scale the verified solution, from 2 DLL instances to possibly 1000 instances (if hw allows).

I can manage to copy the DLL file and maintain the code instance vs. file instance mapping, however to prepare for 100 dll code-instances, i'd need to prepare 100 copies of .cs header file, which is unpractical and ugly. The header file has circa 30 structs and 60 interface methods.

Snippet of header file

// ExternalLibrary.cs
public class ExternalLibrary {
  // Changing this (eg. "ExternalLibrary2.dll") along with class name (ExternalLibrary2) and filename (ExternalLibrary2.cs) is enough, nothing else imho needs to be changed
  public const String ApiDll = "ExternalLibrary.dll";

  [DllImport(ApiDll, CallingConvention = CallingConvention.Cdecl)]
  public static extern Int32 ExternalRoutine(UInt32 Input, out UInt32 Output);
}
  1. Is there any possibility to dynamically create the class (header) instance using different filename, that would affect the [DllImport] annotations?
  2. If not, what scalable solution could be used ?
Marek Sebera
  • 39,650
  • 37
  • 158
  • 244
  • 2
    You can't hope to do this using DllImport. Use LoadLibrary and GetProcAddress and load and bjnd the library dynamically. – David Heffernan Jul 11 '21 at 15:55
  • Hoped for solution that would minimize the changes needed in 3rd party code (the header file is provided periodically by 3rd party), I also have the C header, PAS (Delphi, Pascal) and library as .lib file, if that'd change the possibilities. But mostly DLL+C/CS file is delivered, so I hoped for solution, that would be applicable to different libraries as well. If LoadLibrary/GetProcAddress is the only way, i'll have to write the delegates for 60 different methods, which does not seem to be the most efficient way, but thank you anway – Marek Sebera Jul 11 '21 at 17:52
  • The most efficient way would be to use a single instance of the dll – David Heffernan Jul 11 '21 at 23:20
  • I've since verified that switching to LoadLibrary/GetProcAddress is working, and i'll post a writeup as an answer here, and in this particular case, instead of solving the IPC overhead, i'll be more happy with the proposed solution instead, since the library itself is managing some background processing threads and whole self-lifecycle, using single instance is not viable, thank you anyway – Marek Sebera Jul 12 '21 at 14:48

1 Answers1

1

Thanks to @David Heffernan and few others i've came up with solution, that might work for differently built libraries as well, so i'm sharing it.

This relies heavily on three important things

  1. Library must be loaded each time from separate copy of DLL file
  2. ExternalLibrary.dll is implemented so it can live in it's own directory, not poluting other parts of project distribution files
  3. FunctionLoader here presented is only Windows-compatible (uses Kernel32.dll provided routines)

Solution was tested on top of Microsoft.NET.Sdk.Web sdk and net5.0 target, project was deployed using Kestrel not IIS, so please be aware of it, when testing.

1. Include template directory structure and original library in distribution

Having file structure like this

/RootSolution.sln
/Project/
/Project/Project.csproj
/Project/ExternalLibrary/1/ExternalLibrary.dll

Allows to create per-instance (guid identified) directories like this

/Project/ExternalLibrary/1/ExternalLibrary.dll
/Project/ExternalLibrary/516bbd6d-a5ec-42a5-93e0-d1949ca60767/ExternalLibrary.dll
/Project/ExternalLibrary/6bafaf3c-bc2b-4a1f-ae5c-696c37851b22/ExternalLibrary.dll
/Project/ExternalLibrary/0d0589fc-fc37-434d-82af-02e17a26d927/ExternalLibrary.dll

2. Transform original library header file

Starting with original library header file that looks like this:

// ExternalLibrary.cs
public class ExternalLibrary {
  public const String ApiDll = "ExternalLibrary.dll";

  [DllImport(ApiDll, CallingConvention = CallingConvention.Cdecl)]
  public static extern Int32 ExternalRoutine(UInt32 Input, out UInt32 Output);
}

Transforming it into

// ExternalLibrary.cs
using System;
using System.IO;

public class ExternalLibrary {
  public string LibraryName { get; }
  public Guid InstanceNo { get; }
  public string DefaultLibrarySource { get; } = Path.Combine("ExternalLibrary", "1", "ExternalLibrary.dll");
  public string LibrarySourceTemplate { get; } = Path.Combine("ExternalLibrary", "{0}", "ExternalLibrary.dll");

  public ExternalLibrary(Guid InstanceNo)
  {
    // use constructor provided Guid to construct full path to copy of library and it's living directory
    LibraryName = String.Format(LibrarySourceTemplate, InstanceNo);
    LibraryName = Path.GetFullPath(LibraryName);

    InstanceNo = InstanceNo;

    // create guid-appropriate directory if it does not exist
    var dirName = Path.GetDirectoryName(LibraryName);
    if (!Directory.Exists(dirName))
    {
        Directory.CreateDirectory(dirName);
    }

    // copy over the source library if it's not yet present in guid-appropriate directory
    if (!File.Exists(LibraryName))
    {
        File.Copy(DefaultLibrarySource, LibraryName);
    }

    // load function from correct DLL file into exposed delegated routine
    ExternalRoutine = FunctionLoader.LoadFunction<_ExternalRoutine>(LibraryName, "ExternalRoutine");
  }

  [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
  public delegate Int32 _ExternalRoutine(UInt32 Input, out UInt32 Output);
  public _ExternalRoutine ExternalRoutine;
}

3. Include FunctionLoader class with your project

// FunctionLoader.cs
using System;
using System.Collections.Concurrent;
using System.IO;
using System.Runtime.InteropServices;

/// <summary>
/// Helper function to dynamically load DLL contained functions on Windows only
/// </summary>
internal class FunctionLoader
{
    [DllImport("Kernel32.dll", CharSet = CharSet.Ansi)]
    private static extern IntPtr LoadLibrary(string path);

    [DllImport("Kernel32.dll", CharSet = CharSet.Ansi)]
    private static extern IntPtr GetProcAddress(IntPtr hModule, string procName);

    /// <summary>
    /// Map String (library name) to IntPtr (reference from LoadLibrary)
    /// </summary>
    private static ConcurrentDictionary<string, IntPtr> LoadedLibraries { get; } = new ConcurrentDictionary<string, IntPtr>();

    /// <summary>
    /// Load function (by name) from DLL (by name) and return its delegate
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="dllPath"></param>
    /// <param name="functionName"></param>
    /// <returns></returns>
    public static T LoadFunction<T>(string dllPath, string functionName)
    {
        // normalize
        if (!Path.IsPathFullyQualified(dllPath))
        {
            dllPath = Path.GetFullPath(dllPath);
        }
        // Get preloaded or load the library on-demand
        IntPtr hModule = LoadedLibraries.GetOrAdd(
            dllPath,
            valueFactory: (string dllPath) =>
            {
                IntPtr loaded = LoadLibrary(dllPath);
                if (loaded == IntPtr.Zero)
                {
                    throw new DllNotFoundException($"Library not found in path {dllPath}");
                }
                return loaded;
            }
        );
        // Load function
        var functionAddress = GetProcAddress(hModule, functionName);
        if (functionAddress == IntPtr.Zero)
        {
            throw new EntryPointNotFoundException($"Function {functionName} not found in {dllPath}");
        }
        // Return delegate, casting is hack-ish, but simplifies usage
        return (T)(object)(Marshal.GetDelegateForFunctionPointer(functionAddress, typeof(T)));
    }
}


Please let me know, if this solution worked for you or if you found more elegant way, thank you

Marek Sebera
  • 39,650
  • 37
  • 158
  • 244