How to take a collection of bytes and pull typed values out of it?

Question

Say I have a collection of bytes

var bytes = new byte[] {0, 1, 2, 3, 4, 5, 6, 7};

and I want to pull out a defined value from the bytes as a managed type, e.g. a ushort. What is a simple way to define what types reside at what location in the collection and pull out those values?

One (ugly) way is to use System.BitConverter and a Queue or byte[] with an index and simply iterate through, e.g.:

int index = 0;
ushort first = System.BitConverter.ToUint16(bytes, index);
index += 2; // size of a ushort
int second = System.BitConverter.ToInt32(bytes, index);
index += 4;
...

This method gets very, very tedious when you deal with a lot of these structures!

I know that there is the System.Runtime.InteropServices.StructLayoutAttribute which allows me to define the locations of types inside a struct or class, but there doesn't seem to be a way to import the collection of bytes into that struct. If I could somehow overlay the struct on the collection of bytes and pull out the values, that would be ideal. E.g.

Foo foo = (Foo)bytes; // doesn't work because I'd need to implement the implicit operator
ushort first = foo.first;
int second = foo.second;
...
[StructLayout(LayoutKind.Explicit, Size=FOO_SIZE)]
public struct Foo  {
    [FieldOffset(0)] public ushort first;
    [FieldOffset(2)] public int second;
}

Any thoughts on how to achieve this?

[EDIT: See also my question on how to deal with the bytes when they are big endian.]

Are all the types you are interested integral (or "fixed" arrays of integrals)? — gooch, Mar 18 '10 at 17:20
@gooch: No, but we could settle for that and write helper methods for when we have strings. — Pat, Mar 18 '10 at 17:50
Ok, this works quite well for integral types. Its a bit cumbersome for arrays of these types, as they need to be designated "fixed" (which essentially mean it is a pointer to the beginning of memory). We have not been able to achieve other types, but there is a way to specify Size for each field, this gets hairy quickly. — gooch, Mar 18 '10 at 18:13

gooch · Accepted Answer · 2010-03-18T17:34:13.083

1

We have done this quite a bit as we talk directly to hardware via bytes over serial.

Given the struct definition

[StructLayout(LayoutKind.Explicit, Size=FOO_SIZE)]
public struct Foo  {
    [FieldOffset(0)] public ushort first;
    [FieldOffset(2)] public int second;
}

You can use a class like this to perform the conversion

public class ByteArrayToStruct<StructType>
{
    public StructType ConvertToStruct(int size, byte[] thebuffer)
    {
        try
        {
            int theSize = size;
            IntPtr ptr1 = Marshal.AllocHGlobal(theSize);
            Marshal.Copy(thebuffer, 0, ptr1, theSize);
            StructType theStruct = (StructType)Marshal.PtrToStructure(ptr1, typeof(StructType));
            Marshal.FreeHGlobal(ptr1);
            return theStruct;
        }
        catch (Exception)
        {
            return default(StructType);
        }
    }
}

Conversely, you could also create List from the array and do something like the following:

ushort first = BitConverter.ToInt16(myList.ToArray(), 0);
myList.RemoveRange(0, sizeof(ushort));
[...]

This would essentially be keeping the relevant data at the "head" of the list, so you wont have to keep track of the position in the array.

edited Mar 18 '10 at 17:34

answered Mar 18 '10 at 17:28

gooch

575
5
14

Nice! That's what I was looking for, at least from my preliminary tests. – Pat Mar 18 '10 at 17:57
Oh no, it only works for little-endian bytes (since my host machine is little endian)! Do you have a workaround for taking in big-endian bytes? Note, it is not sufficient to reverse the whole byte array, since it is each value that has swapped endianness: e.g. from `{1, 0, 2, 0, 0, 0}`, you could pull out a `ushort` of `1` and `uint` of `2`, but reversing the bytes would give you a `ushort` of `0` and a `uint` of `0x00020001` or `131073`. – Pat Mar 18 '10 at 22:25
Basically, I need the struct to act as though it were big-endian when it receives the bytes, then to allow me to get the values out of it as though it were the normal endianness of .NET managed code. – Pat Mar 18 '10 at 22:34
I see what you mean. If you are not opposed to using a 3rd party library, I found this SO post: http://stackoverflow.com/questions/217980/c-little-endian-or-big-endian) – gooch Mar 18 '10 at 23:20
I am not opposed to using a library, but Jon's code does not address how to make the struct act as big endian. His code simply adds big-endian functions to System.BitConverter (which I have already done in my own library [too bad I couldn't find Jon's MiscUtil class before now!]). Thanks for searching, though. – Pat Mar 19 '10 at 19:51
Well, I'll give you the credit, @gooch, since you answered my question for the little-endian case and provided the keywords I needed to continue searching. Thanks! – Pat Mar 19 '10 at 20:29
I am sorry that the solution did not satisfy your needs. This is a relatively painless way to stuff bytes into structs. Unfortunately, we have not worked with the Big-Endian -> Little-Endian scenario. That sounds especially tricky as each value needs to be swapped (cannot run a blanket swap). – gooch Mar 22 '10 at 21:05

score 0 · Answer 2 · answered Mar 18 '10 at 16:40

do it the first way. define a Buffer class that has the data bytes and a cursor. THen define methods like getInt16, getInt32 etc. Then you go

  Buffer b(bytes);
  ushort a = b.getInt16();
  int x = b.getInt32();

I have this in my utils bag. I also have the opposite, to make a buffer out of ints strings,....

How to take a collection of bytes and pull typed values out of it?

2 Answers2