How to pass a structure data from C to C# application on the same system with named pipes

Question

I have data in the following structure in C:

struct data
{
    unsigned int count;
    unsigned int total;
};

I am able to send string data from C to C# using namedpipe. But now I need to send this structure.

Can anyone help me pass this information from C to C# using namedpipes?

Here is a sample C program I referred:

#include <windows.h>
#include <stdio.h>

HANDLE fileHandle;

void ReadString(char* output) {
  ULONG read = 0;
  int index = 0;
  do {
    ReadFile(fileHandle, output + index++, 1, &read, NULL);
  } while (read > 0 && *(output + index - 1) != 0);
}

int main()
{
  // create file
  fileHandle = CreateFileW(TEXT("\\\\.\\pipe\\my-very-cool-pipe-example"), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);

  // read from pipe server
  char* buffer = new char[100];
  memset(buffer, 0, 100);
  ReadString(buffer);

  printf("read from pipe server: %s\r\n", buffer);

  // send data to server
  const char* msg = "hello from C\r\n";
  WriteFile(fileHandle, msg, strlen(msg), nullptr, NULL);
}

Here is the corresponding C# program:

using System;
using System.IO;
using System.IO.Pipes;
using System.Threading.Tasks;

namespace PipeServerCsharp
{
    class Program
    {
        static void Main(string[] args)
        {
          var instances = new Task[5];

          for (int i = 0; i < instances.Length; i++)
          {
            instances[i] = Task.Run(() =>
            {
              var namedPipeServer = new NamedPipeServerStream("my-very-cool-pipe-example", PipeDirection.InOut, 5, PipeTransmissionMode.Byte);
              var streamReader = new StreamReader(namedPipeServer);
              namedPipeServer.WaitForConnection();

              var writer = new StreamWriter(namedPipeServer);
              writer.Write("Hello from c#");
              writer.Write((char)0);
              writer.Flush();
              namedPipeServer.WaitForPipeDrain();

              Console.WriteLine($"read from pipe client: {streamReader.ReadLine()}");
              namedPipeServer.Dispose();
            });
          }

          Task.WaitAll(instances);
        }
    }
}

Taken from: https://github.com/gabbersepp/dev.to-posts/tree/master/blog-posts. IPC example

Answer 1

Intro

A pipe has no concept of data or structure. It is simply bytes in at one end and bytes out at the other end. So what is the best way to convert a C struct into a stream of bytes and then into a C# struct which for all programming purposes identical to C struct?

Struct Background

Obviously a struct is a block of memory on the C program side so the simplest way is just to copy the struct byte for byte (hypothetically) and stick them into the input side of the pipe. A struct in C# is also just a block of memory although it has some additional data stored for the .NET runtime. Read from the pipe byte for byte into the C# struct

First a word on structs. While a struct might be declared as

struct data
{
    __int16 count;
    __int32 total;
    __int16 average;
};

the compiler is free to pack the struct. Most compilers will probably pack the above struct into (these are just examples)

struct data
{
    __int16 count;
    __int16 count_padding; // some padding
    __int32 total;
    __int16 average;
    __int16 average_padding; // some padding
};

and obviously change the rest of your code accordingly. CPUs fetch on memory on boundaries. On a 32 bit system having to fetch the __int32 may require 2 fetches if its not aligned on a boundary. So your compiler will do the hard work and rearrange your struct. The problem comes in is different compilers or the same compiler with different optimization settings will create different struct which mean a different layout in memory. [See Structure padding and packing for some more details]

If we feed one struct as is in bytes into a pipe and a different program reads it where it altered the struct differently then their will be a problem. So keep struct packing in mind. You could reorder the above struct as follows so everything aligns "better"

struct data
{
    __int16 count;   // rearranged
    __int16 average;
    __int32 total;
};

There are special compiler options to force packing.

Example

Without further aud here is a working example. Run the C# program first and then the C program.

Note: I have specially chosen to #pragma pack(push, 4) and [StructLayout(LayoutKind.Sequential, Pack = 32)] to demonstrate how you might influence packing. If you leave out the #pragma pack(push, 4) and Pack = 32 it will work too or choose some other matching pair. [StructLayout(LayoutKind.Sequential] is important as it prevents reorder the struct's fields of the struct. C doesn't reorder structs (AFAIK) and only pads so that's why there is no option there.

using System;
using System.IO;
using System.IO.Pipes;
using System.Runtime.InteropServices;

using var namedPipeServer = new NamedPipeServerStream("my-very-cool-pipe-example", PipeDirection.InOut, 5, PipeTransmissionMode.Byte);
// var streamReader = new StreamReader(namedPipeServer); don't want a stream reader since it reads text (We need binary)
var binaryReader = new BinaryReader(namedPipeServer);
var snapshot = new Snapshot();
var buffer = new byte[16].AsSpan(); // make sure your buffer is big enough

namedPipeServer.WaitForConnection(); // wait for the C program to connect

int cc = 0;
while(namedPipeServer.IsConnected && ++cc < 5)
{
    snapshot.Count = binaryReader.ReadInt16();
    snapshot.Average = binaryReader.ReadInt16();
    snapshot.Total = binaryReader.ReadInt32();
    // tada the struct is populated

    Console.WriteLine(snapshot);
}

Console.WriteLine("Now using MemoryMarshal");

while (namedPipeServer.IsConnected)
{
    var bytesRead = binaryReader.Read(buffer);
    var snapshots = MemoryMarshal.Cast<byte, Snapshot>(buffer);

    // tada the struct is populated
    Console.WriteLine($"BytesRead = {bytesRead}, {snapshots[0]}");
}

// Same C struct
[StructLayout(LayoutKind.Sequential /* don't reorder */, Pack = 32)] // 32 bit boundaries like the C version (usually leave 0 / default)
public struct Snapshot
{
    public Int16 Count;
    public Int16 Average;
    public Int32 Total;

    public override string ToString() => 
        $"Snapshot {{Count: {Count}, Average: {Average}, Total: {Total}}}";
}

The C program

#include <windows.h>
#include <stdio.h>
#include <inttypes.h>

#pragma pack(push, 4) // align to 32 bit boundaries for example sake (usually leave default)
typedef struct tagSnapshot {
    __int16        Count;
    __int16        Average;
    __int32        Total;
} Snapshot;
#pragma pack(pop) // undo the align we don't want the whole program to have our change

int main()
{
    HANDLE hPipe;
    Snapshot snapshot;

    // Create Write Only Pipe
    hPipe = CreateFile(TEXT("\\\\.\\pipe\\my-very-cool-pipe-example"), GENERIC_WRITE, FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);

    snapshot.Total = 0;

    for (short i = 0, cc = 0; cc < 10; i+=3, cc++) {

        // some random maths (to change the struct)
        snapshot.Average = cc / 2; 
        snapshot.Count = cc;
        snapshot.Total += cc;

        // struct is now in the pipe as a stream of bytes
        Sleep(200);
        WriteFile(hPipe, &snapshot, sizeof(Snapshot), NULL, NULL);

        printf("Snapshot {Count: %d, Average: %d, Total: %d}\n", snapshot.Count, snapshot.Average, snapshot.Total);
    }

    CloseHandle(hPipe);
}

Conclusion

Sending structs is relatively trivial especially if its only one or some repeating pattern. It becomes more tricky if there is more complex coordination required and different structs being sent. In that case using or construct a library to abstract the complex becomes required. That said Win32 NamedPipes have a message format which might help depending on your needs.

Answer 2

You can alwasy serialize and deserialize.

Serialization in C# is the process of bringing an object into a form that it can be written on stream. It's the process of converting the object into a form so that it can be stored on a file, database, or memory; or, it can be transferred across the network. Its main purpose is to save the state of the object so that it can be recreated when needed.

serialization

As the name suggests, deserialization in C# is the reverse process of serialization. It is the process of getting back the serialized object so that it can be loaded into memory. It resurrects the state of the object by setting properties, fields etc.

Answer 3

A simple method that has worked for me often, is to have the the first 4 bytes denote the size of the data.

So, you have

struct data
{
    unsigned int count;
    unsigned int total;
};

You can define -

struct package {
int sz,
data Data;
};

Send this as a byte stream. On the other end, you have a byte buffer.

unsigned char* recv;
... // Receive data into recv
...
...

int sz = *((int *) recv);

data Data ;
memcpy (&Data, recv+sizeof(int), sizeof(data));

This method will work for all size data and all types of data structures. I have shown C code but the concept will work for C#.