Using a dynamically loaded dll that contains a dynamically loaded dll itself

Question

Intro

My program allows other developers to write plugins. These plugins(which are dll files) are placed in a certain directory and my program is able to load them at runtime like this:

Assembly assembly = Assembly.LoadFrom("MyPluginPath");
object Plugin = assembly.CreateInstance("Plugins.Main", true);

Then the program will be able to call the main method from the plugin this way:

var pluginType = Plugin.GetType();
var methodOutput = pluginType.GetMethod("MyInternalMethod").Invoke(Plugin, new object[] {param});

This works fine for all the plugins I've wrote so far that followed a certain structure as below:

namespace Plugins
{
    public class Main : IDisposable
    {
        public Bitmap MyInternalMethod(Bitmap param)
        {
             // method body
        }
    }
}

Problem

Now there is this new plugin I've to wrote which loads an external dll at runtime itself:

namespace Plugins
{
    public class Main : IDisposable
    {
        [System.Runtime.InteropServices.DllImport("MyExternalDLL.dll")]
        unsafe private static extern int Initialize(int* arg1, char* arg2);

        public Bitmap MyInternalMethod(Bitmap param)
        {
             // method body
             Initialize(x,y);
        }
    }
}

Initialize(x,y) method in the above code gives me an error that says

Failed to initialize com library.

Notes

1. I created an executable test application for my plugin and it worked fine, so I know the problem occurs when using it as a dll at runtime.
2. I thought maybe I should load my external dll in the main program and then pass it as a Func<> object to my plugin but Func<> doesn't allow pointer variables (Initialize method contains arguments like int* and char*) and even if it does, I'm not sure if this solution is gonna work.
3. I tried using winApi as "Sinatr" suggested in the comments by following this answer but the error stays. I even tried loading MyExternalDLL.dll in the main program using winApi and passing the retrieved address of Initialize method to my plugin like the following code but the error remains the same (this way the error happens on func(x,y)):

Main program:

    [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr LoadLibrary(string name);

    [DllImport("kernel32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
    private static extern IntPtr GetProcAddress(IntPtr hModule, string name);

    IntPtr dllHandle = IntPtr.Zero;
    IntPtr addr = IntPtr.Zero;

    private void RunPlugin()
    {
        dllHandle = LoadLibrary("MyExternalDLL.dll");
        addr = GetProcAddress(dllHandle, "Initialize");
        var methodOutput = pluginType.GetMethod("MyInternalMethod").Invoke(Plugin, new object[] {param, addr});
    }

Plugin:

    public class Main : IDisposable
    {
        private unsafe delegate byte initialize_api(int* arg1, char* arg2);

        public Bitmap MyInternalMethod(Bitmap param, IntPtr addr)
        {
            // method body
            //initialize_api is a delegate with the same signature as Initialize(x,y); 
            var func = (initialize_api)Marshal.GetDelegateForFunctionPointer(addr, typeof(initialize_api));
            func(x,y);
        }
    }

4. Since we are talking about plugins here, I don't think implementing a common Interface which both my program and the plugins can use is possible.

Question

Is it even possible to use a dynamically loaded dll that contains a dynamically loaded dll the way I described? if it is, what should I do to make this work?