0

I recently inherited a program that mixes C++ and C++/CLI (.NET). It interfaces to other components over the network, and to a Driver DLL for some special hardware. However, I am trying to figure out the best way to send the data over the network, as what is used seems non-optimal.

The data is stored in a C++ Defined Structure, something like:

    struct myCppStructure {
        unsigned int field1;
        unsigned int field2;
        unsigned int dataArray[512];
    };

The program works fine when accessing the structure itself from C++/CLI. The problem is that to send it over the network the current code does something like the following:

    struct myCppStructure* data;
    IntPtr dataPtr(data);
    // myNetworkSocket is a NetworkStream cast as a System::IO::Stream^
    System::IO::BinaryWriter^ myBinWriter = gcnew BinaryWriter(myNetworkSocket);
    __int64 length = sizeof(struct myCppStructure) / sizeof(__int64);
    unsigned __int64* ptr = static_cast<__int64*>(dataPtr.toPointer());
    for (unsigned int i = 0; i < (length / sizeof(unsigned __int64)); i++)
        myBinWriter->Write((*ptr)++);

In normal C++ it'd usually be a call like:

    myBinWriter->Write(ptr,length);

But I can't find anything equivalent in C++/CLI. System::IO::BinaryWriter only has basic types and some array<>^ versions of a few of them. Is there nothing more efficient?

P.S. These records are generated many times a second; so doing additional copying (e.g. Marshaling) it out of the question.

Note: The original question asked about C#. I failed to realize that what I was thinking of as C# was really "Managed C++" (aka C++/CLI) under .NET. The above has been edited to replace 'C#' references with 'C++/CLI' references - which I am using for any version of Managed C++, though I am notably using .NET 3.5.

TemporalBeing
  • 311
  • 2
  • 6
  • 1
    Since C# is memory managed, you don't have access to the memory pointer like you do in C++, which means you can't gain direct access to the byte representation of the object to send a byte for byte copy without first creating an accessible byte representation of the object. You'll have to prep the object for transport (i.e. Marshal the data) by converting it to a series of bytes first, then use the BinaryWriter to send it across the network. – Paul Jul 31 '12 at 21:08
  • 1
    Are you sure you need to go down this rabbit hole? Check this out: http://stackoverflow.com/a/619273/347062. – kakridge Jul 31 '12 at 21:11
  • We've considered rewriting it to get rid of C# because it's using 75-80% of a Core2 Duo 2GHz+ processor, and the thunking between C# and C++ is a big contributor. Looking at the code, having loops like what I listed above is also a big contributor - especially when the lower level APIs called by C#/.NET have the ability to send a buffer by simply specifying the length of the buffer (e.g. WinSock, on which NetStream is suppose to be based per the Documentation). It just seems poor language and VM design to support native interfaces but provide no support for them things like this. – TemporalBeing Jul 31 '12 at 21:24
  • That's your best bet for "bit bashing". – kakridge Jul 31 '12 at 21:27
  • @TemporalBeing, instead of turning this into a C++ vs. C# debate, let's see instead what infrastructure IS supported for bit bashing. The (new) solution I've provided below only has one extra memcpy which you can avoid if not using a BinaryWriter, copying the data directly to the socket. – Ani Jul 31 '12 at 22:08
  • @ananthonline - the copying would be too much for performance. – TemporalBeing Aug 01 '12 at 14:18
  • @Paul - I already have the data as a series of bytes. I'm trying to figure out how to write to the network without using a loop, much like I would on any other platform or language - simply give it a pointer and the data length. C++/CLI seems to lack such a basic mechanism. – TemporalBeing Aug 01 '12 at 17:49
  • @TemporalBeing my comment about needing to get the series of bytes first was in regards to how the question was posted originally with going from C++ to C#. Changing it to C++/CLI, you're still dealing with managed memory, and the BinaryWriter methods expect managed memory datatypes. IntPtr is not a managed memory type, but a "platform specific type that is used to represent a pointer or a handle" http://goo.gl/6kfrw If your byte array you already have is managed, then just use myBinWriter->Write(array, index, count) http://goo.gl/OqTjU Otherwise, you'll still have to get a managed byte array – Paul Aug 01 '12 at 19:53

3 Answers3

1

What you want to do is to find out how the C++ struct is packed, and define a struct with the correct StructLayout attribute.

To define the fixed length int[], you can defined a fixed size buffer inside it. Note that to use this you will have to mark your project /unsafe.

Now you're ready to convert that struct to a byte[] using two steps

  1. Pin the array of structs in memory using a GCHandle.Alloc - this is fast and shouldn't be a performance bottleneck.
  2. Now use Marshal.Copy (don't worry, this is as fast as a memcpy) with the source IntPtr = handle.AddrOfPinnedObject.

Now dispose the GCHandle and you're ready to write the bytes using the "Write" overload mentioned by Serg Rogovtsev.

Hope this helps!

Ani
  • 10,826
  • 3
  • 27
  • 46
  • Please note in my original post I said Marshaling was not an option. I am aware of how the data is packed; and it's wrapped in a C# structure already, but not as you describe as far as I can tell. (I'm pretty new to C# so I could be wrong.) – TemporalBeing Jul 31 '12 at 21:07
  • Also, you did't answer my question about how to actually send it on the network itself. System::IO::BinaryWriter uses array<>^ objects, not simply byte[] arrays. – TemporalBeing Jul 31 '12 at 21:12
  • @TemporalBeing I'm sorry, but you're wrong: http://msdn.microsoft.com/en-us/library/bx149a0k.aspx – Sergei Rogovtcev Jul 31 '12 at 21:17
  • @SergRogovtsev - funny you link to exactly one of the functions I was talking about. It uses the array<>^ object, not a byte[] array. – TemporalBeing Jul 31 '12 at 21:25
  • @TemporalBeing it seems to me that you're not using C#, but some other language. Because (a) this link specifically states `public virtual void Write(byte[] buffer)` and (b) there's no type `array<>^` in C#. BTW, *C++* notation for it reads `virtual void Write(array^ buffer)` – Sergei Rogovtcev Jul 31 '12 at 21:27
  • @SergRogovtsev - the code is mixed between C# and C++. The specific file is C++. So perhaps is more of a Managed C++/CLR question than C# question. – TemporalBeing Jul 31 '12 at 21:31
  • 1
    @TemporalBeing Would you please edit your question so it does not mention C# in reference to C++ code? – Sergei Rogovtcev Jul 31 '12 at 21:33
  • @TemporalBeing - I've edited my post to remove any unnecessary marshaling. I only use fast pinning and copy - this method ought to be really fast. – Ani Jul 31 '12 at 21:41
  • @ananthonline - as noted several times in several replies, copying is out of the question. I already have the data in exactly the format I need it. It's just a matter of trying to find the C++/CLI API to actually put it on the wire correctly, and C++/CLI seems to lack such a very basic mechanism - forcing you to go through all kinds of other mechanisms to get there. – TemporalBeing Aug 01 '12 at 17:47
1

Your structure consists of "basic types" and "array of them". Why can't you just wrote them sequentially using BinaryWriter? Something like

binWriter.Write(data.field1);
binWriter.Write(data.field2);
for(var i = 0; i < 512; i++)
    binWriter.Write(data.dataArray[i]);
Sergei Rogovtcev
  • 5,804
  • 2
  • 22
  • 35
  • It's basic types, yes; but a variety of different ones. Also there are several different structures being sent and I'd rather not have different writers for each structure type. Simple is usually better, and in this case C and C++'s very simple method works great and very performant. I can't figure out why C# doesn't have a nice, easy to find equivalent. – TemporalBeing Jul 31 '12 at 21:10
  • It has, in fact. It's called `BinarySerializer`. You have to understand that C# was designed as very high-level language and thus it does not expect any specific memory layout. So memory arithmetic in general does not work there. – Sergei Rogovtcev Jul 31 '12 at 21:16
  • You could probably use reflection to determine the fields at runtime and create a single writer that could handle any type of struct. – Kibbee Aug 01 '12 at 13:16
  • @Kibbee That's what `BinarySerializer` does. – Sergei Rogovtcev Aug 01 '12 at 13:28
  • @SergRogovtsev - from what I can tell from the C++/CLI documentation, BinarySerializer is not available (http://msdn.microsoft.com/en-us/library/ty01x675%28v=vs.90%29.aspx). It is for VB and C#, but not C++. – TemporalBeing Aug 01 '12 at 13:54
  • @TemporalBeing It can't be unavailable, it's part of BCL. There're no samples, but the class itself is there. Although I was wrong and it's called `BinaryFormatter`. – Sergei Rogovtcev Aug 01 '12 at 14:20
  • @SergRogovtsev - I'm not sure BinaryFormatter buys me anything. The instance of the data I'm trying to send is not a System::Object, so it very explicitly lacks some of the requirements for BinaryFormatter. – TemporalBeing Aug 01 '12 at 17:44
  • *Everything* is `System::Object` in CLR. – Sergei Rogovtcev Aug 01 '12 at 17:51
  • @SergRogovtsev - so when you allocate a Native C++ structure, it becomes System::Object? As I noted, it's a mix of C++ (Native) and C++/CLI. The data in question is allocated via C++ new, not gcnew. it's also probably allocated in the native C++ code. So I doubt its System::Object. – TemporalBeing Aug 01 '12 at 19:20
  • Now *that*'s out of my depth, sorry. I'd *suppose* that as soon as it has passed to CLR side, it would become `object`, but I wouldn't eat my hat for that. – Sergei Rogovtcev Aug 01 '12 at 19:32
0

In C# you could do the following. Start by defining the structure.

[StructLayout ( LayoutKind.Sequential )]
internal unsafe struct hisCppStruct
{
    public uint field1;
    public uint field2;
    public fixed uint dataArray [ 512 ];
}

And then write it using the binary writer as follows.

hisCppStruct @struct = new hisCppStruct ();
@struct.field1 = 1;
@struct.field2 = 2;
@struct.dataArray [ 0 ] = 3;
@struct.dataArray [ 511 ] = 4;

using ( BinaryWriter bw = new BinaryWriter ( File.OpenWrite ( @"C:\temp\test.bin") ) )
{
    int structSize = Marshal.SizeOf ( @struct );
    int limit = structSize / sizeof ( uint );

    uint* uintPtr = (uint*) &@struct;

    for ( int i = 0 ; i < limit ; i++ )
        bw.Write ( uintPtr [ i ] );
}

I'm pretty sure you can do exactly the same in managed C++.

Thomas
  • 1,281
  • 8
  • 22
  • I'm trying to eliminated that loop at the end - the bw.Write() in your example. – TemporalBeing Aug 01 '12 at 13:51
  • @TemporalBeing The version I posted does not have a loop. Also - this version looks like it could crash if the GC were to try and relocate the array while the copy is running. :) My pinning solution should stop that. – Ani Aug 01 '12 at 14:18
  • @ananthonline `fixed ( hisCppStruct* ptr = &@struct )` gives the following error `You cannot use the fixed statement to take the address of an already fixed expression` – Thomas Aug 01 '12 at 14:32