11

Scenario

This should be an easy task, but for some reason I can't get it going as intended. I have to marshal a basic C++ struct during a reversed-P/Invoke call (unmanaged calling managed code).

The issue only arises when using bool within the struct, so I just trim the C++ side down to:

struct Foo {
    bool b;
};

Since .NET marshals booleans as 4-byte fields by default, I marshal the native boolean explicitly as a 1 byte-length field:

public struct Foo {
    [MarshalAs(UnmanagedType.I1)] public bool b;
}

When I call an exported managed static method with the following signature and body:

public static void Bar(Foo foo) {
    Console.WriteLine("{0}", foo.b);
}

I get the correct boolean alpha-representation printed. If I extend the structure with more fields, the alignment is correct and the data is not corrupt after marshalling.

Problem

For some reason, if I do not pass this marshalled struct as an argument but rather as a return type by value:

public static Foo Bar() {
    var foo = new Foo { b = true };
    return foo;
}

The application crashes with the following error message:

enter image description here

If I change the managed structure to hold a byte instead of a bool

public struct Foo {
    [MarshalAs(UnmanagedType.I1)] public byte b;
}

public static Foo Bar() {
    var foo = new Foo { b = 1 };
    return foo;
}

the return value is marshalled properly without an error to an unmanaged bool.

I don't unterstand two things here:

  1. Why does a paramter marshalled with bool as described above work, but as a return value give an error?
  2. Why does a byte marshalled as UnmanagedType.I1 work for returns, but a bool also marshalled with UnmanagedType.I1 does not?

I hope my description makes sense -- if not, please let me know so I can change the wording.

EDIT: My current workaround is a managed struct like:

public struct Foo {
    private byte b;
    public bool B {
        get { return b != 0; }
        set { b = value ? (byte)1 : (byte)0; }
}

which honestly, I find quite ridiculous...

EDIT2: Here is an almost-MCVE. The managed assembly has been recompiled with proper symbol exports (using .export and .vtentry attributes in IL code), but there should be no difference to C++/CLI calls. So this code is not working "as-is" without doing the exports manually:

C++ (native.dll):

#include <Windows.h>

struct Foo {
    bool b;
};

typedef void (__stdcall *Pt2PassFoo)(Foo foo);
typedef Foo (__stdcall *Pt2GetFoo)(void);

int main(int argc, char** argv) {
    HMODULE mod = LoadLibraryA("managed.dll");
    Pt2PassFoo passFoo = (Pt2PassFoo)GetProcAddress(mod, "PassFoo");
    Pt2GetFoo getFoo = (Pt2GetFoo)GetProcAddress(mod, "GetFoo");

    // Try to pass foo (THIS WORKS)
    Foo f1;
    f1.b = true;
    passFoo(f1);

    // Try to get foo (THIS FAILS WITH ERROR ABOVE)
    // Note that the managed method is indeed called; the error
    // occurs upon return. If 'b' is not a 'bool' but an 'int'
    // it also works, so there must be something wrong with it
    // being 'bool'.
    Foo f2 = getFoo();

    return 0;
}

C# (managed.dll):

using System;
using System.Runtime.InteropServices;

public struct Foo {
    [MarshalAs(UnmanagedType.I1)] public bool b;
    // When changing the above line to this, everything works fine!
    // public byte b;
}

/*
    .vtfixup [1] int32 fromunmanaged at VT_01
    .vtfixup [1] int32 fromunmanaged at VT_02
    .data VT_01 = int32(0)
    .data VT_02 = int32(0)
*/

public static class ExportedFunctions {
    public static void PassFoo(Foo foo) {
         /*
             .vtentry 1:1
             .export [1] as PassFoo
         */               

         // This prints the correct value, and the
         // method returns without error.
         Console.WriteLine(foo.b);
    }

    public static Foo GetFoo() {
         /*
             .vtentry 2:1
             .export [2] as GetFoo
         */

         // The application crashes with the shown error
         // message upon return.
         var foo = new Foo { b = true; }
         return foo;
    }
}
PuerNoctis
  • 1,364
  • 1
  • 15
  • 34
  • Can we have an MCVE? So we can see the function calls too. – David Heffernan Aug 20 '15 at 06:24
  • @DavidHeffernan I added an MCVE as best I could. As I use reverse P/Invoke mechanisms and manipulate the IL code directly it will not run out-of-the-box without manual changes to the assembly. – PuerNoctis Aug 20 '15 at 06:49
  • Manipulate the IL? Hmm. Surely that's going to be relevant!! – David Heffernan Aug 20 '15 at 06:53
  • @DavidHeffernan The basic concept is described here (http://www.codeproject.com/Articles/37675/Simple-Method-of-DLL-Export-without-C-CLI). You can find this mechanic in several articles on the inernet, but it's too much to explain here in short. But this _should_ (hopefully) not be the problem here as it directly reflects C++/CLI behavior. The only chance that this is an issue IMO would be that `ilasm.exe` generates faulty marshalling thunks - which would be a candidate to be reported to Microsoft Connect for fixing. – PuerNoctis Aug 20 '15 at 06:59
  • Can you avoid using C++ for interop? I've had far better results with sticking to pure C for the interop structures and function pointers. – Luaan Aug 20 '15 at 07:01
  • @Luaan That's not the issue at all. There's no problem with that. – David Heffernan Aug 20 '15 at 07:06
  • @Luaan: Unfortunately not. We have a "loader" for a plugin-like mechanic wrtitten in C++ with respective basic structures (nothing fancy, but with bool...). Only option would be to create an intermediate C++/CLI proxy assembly which I ultmately wanted to bypass by using the reverse P/Invoke. – PuerNoctis Aug 20 '15 at 07:06
  • @Puer I'd test a clean room approach with no IL wackiness. Have the C# host pass a delegate to the C++ and then have the C++ call that delegate. Does that work? If so then the finger points at your own hand in the marshalling. Compare your assembly with one produced by UnmanagedExports. Or one produced by C++/CLI. – David Heffernan Aug 20 '15 at 07:08
  • This is not an exact duplicate, but the problem and the solution is the same - the default marshalling only handles blittable structs for return values; since you're mapping a 4-byte value to a 1-byte value, this no longer works (as the `MarshalDirectiveException` tries to tell us). – Luaan Aug 20 '15 at 08:31
  • @DavidHeffernan Both cases deal with returning non-blittable structures from a marshalled call - the problem is the same with `DllImport` and the export. – Luaan Aug 20 '15 at 10:21
  • @Luaan Hmm, maybe you are right. I think I have mis-interpreted the question. – David Heffernan Aug 20 '15 at 10:29
  • @Luaan Yep, the issue looks quite exactly the same to what I am experiencing, and the answer from Hans sheds some light on the problem. I might just take my workaround with `byte` instead of `bool` as also suggested in the answer of the duplicate. – PuerNoctis Aug 20 '15 at 11:44
  • As a note, while reading random stuff at the internet, I've noticed that by having DLL exports, the assembly is no longer considered a pure managed assembly. This doesn't necessarily mean any trouble for your use case, but it's probably a good idea to be aware of that. – Luaan Aug 21 '15 at 11:27
  • Actually, try reading http://blogs.msdn.com/b/cbrumme/archive/2003/08/20/51504.aspx - it seems to point out a few issues with what you're trying to do. If you want to skip to the first kind of (quite random) problems this can cause, search for `mscorwks.dll`. I'm not sure if all have been fixed since (the article is about .NET 1.0 and 1.1), they certainly tried. – Luaan Aug 21 '15 at 13:13

1 Answers1

10

The underlying problem is the same as with this question - Why DllImport for C bool as UnmanagedType.I1 throws but as byte it works The exception you're getting is MarshalDirectiveException - getting the remaining information about the exception is a bit trickier, but unnecessary.

In short, marshalling for return values only works for blittable structures. When you specify use a boolean field, the structure is no longer blittable (because bool isn't blittable), and will no longer work for return values. This is simply a limitation of the marshaller, and it applies for both DllImport and your attempts at "DllExport".

Quoting the relevant piece of documentation:

Structures that are returned from platform invoke calls must be blittable types. Platform invoke does not support non-blittable structures as return types.

It's not said outright, but the same thing applies when being invoked.

The simplest workaround is to stick with your "byte as a backing field, bool as a property" approach. Alternatively, you could use the C BOOL instead, which will work just fine. And of course, there's always the option of using a C++/CLI wrapper, or even just hiding the real layout of the structure in your helper methods (in this case, your export methods will call another method that deals with the real Foo type, and handle the proper conversion to the Foo++ type).

It's also possible to use a ref argument instead of a return value. This is in fact a common pattern in unmanaged interop:

typedef void(__stdcall *Pt2GetFoo)(Foo* foo);

Foo f2 = Foo();
getFoo(&f2);

on the C++ side, and

public static void GetFoo(ref Foo foo)
{
    foo = new Foo { b = true };
}

on the C# side.

You could also make your own boolean type, a simple struct with a single byte field, with implicit cast operators to and from bool - it's not going to work exactly as a real bool field, but it should work just fine most of the time.

Community
  • 1
  • 1
Luaan
  • 62,244
  • 7
  • 97
  • 116