C# - Cast a byte array to an array of struct and vice-versa (reverse)

Question

I would like to save a Color[] to a file. To do so, I found that saving a byte array to a file using "System.IO.File.WriteAllBytes" should be very efficient.

I would like to cast my Color[] (array of struct) to a byte array into a safe way considering:

• Potential problem of little endian / big endian (I think it is impossible to happen but want to be sure)
• Having 2 differents pointer to the same memory which point to different type. Does the garbage collection will know what to do - moving objects - deleting a pointer ???

If it is possible, it would be nice to have a generic way to cast array of byte to array of any struct (T struct) and vice-versa.

If not possible, why ?

Thanks, Eric

I think that those 2 solutions make a copy that I would like to avoid and also they both uses Marshal.PtrToStructure which is specific to structure and not to array of structure:

• Reading a C/C++ data structure in C# from a byte array
• How to convert a structure to a byte array in C#?

Answer 1

Since .NET Core 2.1, yes we can! Enter MemoryMarshal.

We will treat our Color[] as a ReadOnlySpan<Color>. We reinterpret that as a ReadOnlySpan<byte>. Finally, since WriteAllBytes has no span-based overload, we use a FileStream to write the span to disk.

var byteSpan = MemoryMarshal.AsBytes(colorArray.AsSpan());
fileStream.Write(byteSpan);

As an interesting side note, you can also experiment with the [StructLayout(LayoutKind.Explicit)] as an attribute on your fields. It allows you to specify overlapping fields, effectively allowing the concept of a union.

Here is a blog post on MSDN that illustrates this. It shows the following code:

[StructLayout(LayoutKind.Explicit)]
public struct MyUnion
{
    [FieldOffset(0)]
    public UInt16 myInt;

    [FieldOffset(0)]
    public Byte byte1;

    [FieldOffset(1)]
    public Byte byte2;
}

In this example, the UInt16 field overlaps with the two Byte fields.

This seems to be strongly related to what you are trying to do. It gets you very close, except for the part of writing all the bytes (especially of multiple Color objects) efficiently. :)

Answer 2

Regarding Array Type Conversion

C# as a language intentionally makes the process of flattening objects or arrays into byte arrays difficult because this approach goes against the principals of .NET strong typing. The conventional alternatives include several serialization tools which are generally seen a safer and more robust, or manual serialization coding such as BinaryWriter.

Having two variables of different types point to the same object in memory can only be performed if the types of the variables can be cast, implicitly or explicitly. Casting from an array of one element type to another is no trivial task: it would have to convert the internal members that keep track of things such as array length, etc.

A simple way to write and read Color[] to file

void WriteColorsToFile(string path, Color[] colors)
{
    BinaryWriter writer = new BinaryWriter(File.OpenWrite(path));

    writer.Write(colors.Length);

    foreach(Color color in colors)
    {
        writer.Write(color.ToArgb());
    }

    writer.Close();
}

Color[] ReadColorsFromFile(string path)
{
    BinaryReader reader = new BinaryReader(File.OpenRead(path));

    int length = reader.ReadInt32();

    Colors[] result = new Colors[length];

    for(int n=0; n<length; n++)
    {
        result[n] = Color.FromArgb(reader.ReadInt32());
    }

    reader.Close();
}

Answer 3

You could use pointers if you really want to copy each byte and not have a copy but the same object, similar to this:

var structPtr = (byte*)&yourStruct;
var size = sizeof(YourType);
var memory = new byte[size];
fixed(byte* memoryPtr = memory)
{
    for(int i = 0; i < size; i++)
    {
        *(memoryPtr + i) = *structPtr++;
    }
}
File.WriteAllBytes(path, memory);

I just tested this and after adding the fixed block and some minor corrections it looks like it is working correctly.

This is what I used to test it:

public static void Main(string[] args)
{
    var a = new s { i = 1, j = 2 };
    var sPtr = (byte*)&a;
    var size = sizeof(s);
    var mem = new byte[size];
    fixed (byte* memPtr = mem)
    {
        for (int i = 0; i < size; i++)
        {
            *(memPtr + i) = *sPtr++;
        }
    }
    File.WriteAllBytes("A:\\file.txt", mem);
}

struct s
{
    internal int i;

    internal int j;
}

The result is the following:

(I manually resolved the hex bytes in the second line, only the first line was produced by the program)

Answer 4

    public struct MyX
    {
        public int IntValue;
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 3, ArraySubType = UnmanagedType.U1)]
        public byte[] Array;

        MyX(int i, int b)
        {
            IntValue = b;
            Array = new byte[3];
        }

        public MyX ToStruct(byte []ar)
        {

            byte[] data = ar;//= { 1, 0, 0, 0, 9, 8, 7 }; // IntValue = 1, Array = {9,8,7}
            IntPtr ptPoit = Marshal.AllocHGlobal(data.Length);
            Marshal.Copy(data, 0, ptPoit, data.Length);

            MyX x = (MyX)Marshal.PtrToStructure(ptPoit, typeof(MyX));
            Marshal.FreeHGlobal(ptPoit);

            return x;
        }
        public byte[] ToBytes()
        {
            Byte[] bytes = new Byte[Marshal.SizeOf(typeof(MyX))];
            GCHandle pinStructure = GCHandle.Alloc(this, GCHandleType.Pinned);
            try
            {
                Marshal.Copy(pinStructure.AddrOfPinnedObject(), bytes, 0, bytes.Length);
                return bytes;
            }
            finally
            {
                pinStructure.Free();
            }
        }
    }

    void function()
    {
        byte[] data = { 1, 0, 0, 0, 9, 8, 7 }; // IntValue = 1, Array = {9,8,7}
        IntPtr ptPoit = Marshal.AllocHGlobal(data.Length);
        Marshal.Copy(data, 0, ptPoit, data.Length);

        var x = (MyX)Marshal.PtrToStructure(ptPoit, typeof(MyX));
        Marshal.FreeHGlobal(ptPoit);

        var MYstruc = x.ToStruct(data);


        Console.WriteLine("x.IntValue = {0}", x.IntValue);
        Console.WriteLine("x.Array = ({0}, {1}, {2})", x.Array[0], x.Array[1], x.Array[2]);
    }

Answer 5

Working code for reference (take care, I did not need the alpha channel in my case):

// ************************************************************************
// If someday Microsoft make Color serializable ...
    //public static void SaveColors(Color[] colors, string path)
    //{
    //  BinaryFormatter bf = new BinaryFormatter();
    //  MemoryStream ms = new MemoryStream();
    //  bf.Serialize(ms, colors);
    //  byte[] bytes = ms.ToArray();
    //  File.WriteAllBytes(path, bytes);
    //}

// If someday Microsoft make Color serializable ...
    //public static Colors[] LoadColors(string path)
    //{
    //  Byte[] bytes = File.ReadAllBytes(path);
    //  BinaryFormatter bf = new BinaryFormatter();
    //  MemoryStream ms2 = new MemoryStream(bytes);
    //  return (Colors[])bf.Deserialize(ms2);
    //}

    // ******************************************************************
    public static void SaveColorsToFile(Color[] colors, string path)
    {
        var formatter = new BinaryFormatter();

        int count = colors.Length;

        using (var stream = File.OpenWrite(path))
        {
            formatter.Serialize(stream, count);

            for (int index = 0; index < count; index++)
            {
                formatter.Serialize(stream, colors[index].R);
                formatter.Serialize(stream, colors[index].G);
                formatter.Serialize(stream, colors[index].B);
            }
        }
    }

    // ******************************************************************
    public static Color[] LoadColorsFromFile(string path)
    {
        var formatter = new BinaryFormatter();

        Color[] colors;

        using (var stream = File.OpenRead(path))
        {
            int count = (int)formatter.Deserialize(stream);
            colors = new Color[count];

            for (int index = 0; index < count; index++)
            {
                byte r = (byte)formatter.Deserialize(stream);
                byte g = (byte)formatter.Deserialize(stream);
                byte b = (byte)formatter.Deserialize(stream);

                colors[index] = Color.FromRgb(r, g, b);
            }
        }

        return colors;
    }

    // ******************************************************************