C# Unions and array copy

Question

I am trying to copy an array of Struct1 into an array of Struct2 (same binary representation) the fastest way possible. I have defined an union to convert between Struct1[] and Struct2[] but when I call Array.Copy I get an exception saying the array is the wrong type. How can I circumvent that? Buffer.BlockCopy only accepts primitive types. Here is the code:

[StructLayout(LayoutKind.Explicit)]
public struct Struct12Converter
{
    [FieldOffset(0)]
    public Struct1[] S1Array;
    [FieldOffset(0)]
    public Struct2[] S2Array;
}

public void ConversionTest()
{
    var s1Array = new{new Struct1()}
    var converter = new Struct12Converter{S1Array = s1Array};
    var s2Array = new Struct2[1];
    Array.Copy(converter.S2Array,0,s2Array,0,1) //throws here
    //if you check with the debugger, it says converter.S2Array is a Struct1[], 
    //although the compiler lets you use it like a Struct2[]
    //this has me baffled as well.
}

To give more details: I wanted to experiment to see if working with a mutable struct and changing the value of its fields had different performance characteristics compared to working all the time using the same immutable struct. I think it should be similar but I thought it was worth a measurement. The underlying application would be a low latency socket library where I currently use ArraySegment<byte>-based socket APIs. It so happens that in the SocketAsyncEventArgs api, setting the BufferList property triggers an array copy which is where my 'experiment' fails (i have an array of MutableArraySegment which I cannot convert to ArraySegment[] through the same method as before, thus making my comparison pointless).

Answer 1

If the Structs are exactly the same, you can accomplish this by using Marshal.PtrToStructure method.

You need to obtain a pointer to your struct, and then you can "deserialize" it back into another struct (which should have the EXACT same layout).

You can see an example here.

Hope this helps, Ofir.

Answer 2

Are you aware that you have broken the type-system by unsafely treating a Struct1[] as a Struct2[]? This puts the CLR into an undefined state. It can assume that variables of type Struct1[] indeed point to an instance of Struct1[]. You can see pretty much any weird behavior now. (This is not a security issue. This code is not verifiable and required full trust.)

In other words, you have not converted the array contents but obtained a converted object reference.

Copying an array of blittable objects is usually done in the fastest way using memcpy. A manual copy-loop is equivalent to that but I would not trust the JIT to optimize it into a memcpy. The JIT only does the basic optimizations as of the current version.

Answer 3

This code is deliberately unsafe (since what you want to do is unsafe, AFAIK the CLR/JIT can re-order structs for performance reasons)

Also note that the signature for MemCpy can change depending on framework version (it is internal after all)

For performance reasons you should cache that delegate properly

Idea from this question here

    unsafe delegate void MemCpyImpl(byte* src, byte* dest, int len);

    static MemCpyImpl memcpyimpl;

    public unsafe static void Copy(void* src, void* dst, int count)
    {
        byte* source = (byte*)src;
        byte* dest = (byte*)dst;
        memcpyimpl(source, dest, count);
    }

Then forcing your arrays to be byte arrays (actually void* but never mind the detail)

    public static void ConversionTest()
    {
        var bufferType = typeof(Buffer);

        unsafe
        {
            var paramList = new Type[3] { typeof(byte*), typeof(byte*), typeof(int) };
            var memcpyimplMethod = bufferType.GetMethod("Memcpy", BindingFlags.Static | BindingFlags.NonPublic, null, paramList, null);

            memcpyimpl = (MemCpyImpl)Delegate.CreateDelegate(typeof(MemCpyImpl), memcpyimplMethod);
        }

        Struct1[] s1Array = { new Struct1() { value = 123456789 } };
        var converter = new Struct12Converter { S1Array = s1Array };
        var s2Array = new Struct2[1];
        unsafe
        {
            fixed (void* bad = s2Array)
            {
                fixed (void* idea = converter.S2Array)
                {
                    Copy(bad, idea, 4);
                }
            }
        }
    }

Answer 4

Warning: This might be a dangerous trick, since it does circumvent the type system and the assembly would not be verifiable. Still - some superficial testing didn't raise any obvious problems, and for your 'experiment' it might be worth a go. Check through the warnings by @usr in the comments, though...

Under your assumptions (if you can tolerate an unverifiable output assembly) you need no Marshal.XXX, Array.Copy or memcpy at all. You can read the values from the union type as either a Struct1 array or a Struct2 array. My guess, though I have no evidence to back it up, is that the runtime and GC won't notice the discrepancy between the array type and how you're using the elements.

Here's a standalone example that will run in LinqPad. The default packing means you don't actually need the LayoutKind and FieldOffset annotations in Struct1 and Struct2 (though of course you do in the union type Struct12Converter), but it helps to shows this explicitly.

[StructLayout(LayoutKind.Explicit)]
public struct Struct1
{
    [FieldOffset(0)]
    public int Int1;
    [FieldOffset(4)]
    public int Int2;
}

[StructLayout(LayoutKind.Explicit)]
public struct Struct2
{
    [FieldOffset(0)]
    public long Long;
}

[StructLayout(LayoutKind.Explicit)]
public struct Struct12Converter
{
    [FieldOffset(0)]
    public Struct1[] S1Array;
    [FieldOffset(0)]
    public Struct2[] S2Array;
}


public void ConversionTest()
{
    var int1 = 987;
    var int2 = 456;
    var int3 = 123456;
    var int4 = 789123;

    var s1Array = new[] 
    { 
        new Struct1 {Int1 = int1, Int2 = int2},
        new Struct1 {Int1 = int3, Int2 = int4},
    };

    // Write as Struct1s
    var converter = new Struct12Converter { S1Array = s1Array };

    // Read as Struct2s
    var s2Array = converter.S2Array;

    // Check: Int2 is the high part, so that must shift up
    var check0 = ((long)int2 << 32) + int1;
    Debug.Assert(check0 == s2Array[0].Long);
    // And check the second element
    var check1 = ((long)int4 << 32) + int3;
    Debug.Assert(check1 == s2Array[1].Long);

    // Using LinqPad Dump:
    check0.Dump();
    s2Array[0].Dump();

    check1.Dump();
    s2Array[1].Dump();

}

void Main()
{
    ConversionTest();
}