Read an array of structs in C#

Question

I've seen here , and also googling for "marshal" several ways to convert a byte array to a struct.

But what I'm looking for is if there is a way to read an array of structs from a file (ok, whatever memory input) in one step?

I mean, load an array of structs from file normally takes more CPU time (a read per field using a BinaryReader) than IO time. Is there any workaround?

I'm trying to load about 400K structs from a file as fast as possible.

Thanks

pablo

Are you reading them from one big file or 400k small files? If you have all in one file reading should be quite fast I think — Rune Grimstad, Feb 11 '09 at 08:32

score 1 · Answer 1 · edited Aug 07 '12 at 17:05

Following URL may be of interest to you.

http://www.codeproject.com/KB/files/fastbinaryfileinput.aspx

Or otherwise I think of pseudo code like the following:

readbinarydata in a single shot and convert back to structure..

public struct YourStruct
{ 
    public int First;
    public long Second;
    public double Third;
}

static unsafe byte[] YourStructToBytes( YourStruct s[], int arrayLen )
{
    byte[] arr = new byte[ sizeof(YourStruct) * arrayLen ];
    fixed( byte* parr = arr )
    { 
        * ( (YourStruct * )parr) = s; 
    }
    return arr;
}

static unsafe YourStruct[] BytesToYourStruct( byte[] arr, int arrayLen )
{
    if( arr.Length < (sizeof(YourStruct)*arrayLen) )
        throw new ArgumentException();
    YourStruct s[];
    fixed( byte* parr = arr )
    { 
        s = * ((YourStruct * )parr); 
    }
    return s;
}

Now you can read bytearray from the file in a single shot and convert back to strucure using BytesToYourStruct

Hope you can implement this idea and check...

Unfortunately they do it "one by one" here, which is exactly what I want to avoid — , Feb 11 '09 at 08:27
The fixed solution seems to be what I was looking for! thanks! — , Feb 11 '09 at 11:59
Didn't compile on Mono: "Cannot implicitly convert type 'YourStruct[]' to YourStruct" — user666412, Dec 12 '14 at 14:53

score 0 · Answer 2 · edited May 23 '17 at 12:30

0

I found a potential solution at this site - http://www.eggheadcafe.com/software/aspnet/32846931/writingreading-an-array.aspx

It says basically to use Binary Formatter like this:

FileStream fs = new FileStream("DataFile.dat", FileMode.Create); BinaryFormatter formatter = new BinaryFormatter(); formatter.Serialize(fs, somestruct);

I also found two questions from this site - Reading a C/C++ data structure in C# from a byte array and How to marshal an array of structs - (.Net/C# => C++)

I haven't done this before, being a C# .NET beginner myself. I hope this solution helps.

edited May 23 '17 at 12:30

Community

1
1

answered Feb 11 '09 at 08:24

batbrat

5,155
3
32
38

Thanks, but "Serialize" is slow compared to direct memory access, and the other links use "PtrToStructure" to copy "one by one" which is exactly what I want to avoid. – Feb 11 '09 at 08:28
Sorry pablo, I had computer trouble yesterday. So I couldn't follow up. I am not sure how to achieve what you want. All the best with that. – batbrat Feb 12 '09 at 08:19
I hope you find out. If you do, please post an answer here. Thanks. – batbrat Feb 12 '09 at 08:19

Read an array of structs in C#

2 Answers2