Direct json to StructureMap serialized byte[]

Question

I have the following JSON (received as byte[] from a web socket):

{
    "ev": "T",              // Event Type
    "sym": "MSFT",          // Symbol Ticker
    "x": 4,                 // Exchange ID
    "i": "12345",           // Trade ID
    "z": 3,                 // Tape ( 1=A 2=B 3=C)
    "p": 114.125,           // Price
    "s": 100,               // Trade Size
    "c": [0, 1],           // Trade Conditions
    "t": 1536036818784      // Trade Timestamp ( Unix MS )
}

And a structure:

struct Trade
{
    [JsonPropertyName("sym")]
    public string  Symbol { get; set; }
    [JsonPropertyName("p")]
    public decimal Price { get; set; }
    ...
}

Currently I am going from JSON to structure (using the amazing Utf8Json lib), then from structure to bytes[].

I wish to go straight from JSON to MessagePack serialized byte object - as the middle step seems a little redundant.

How can one achieve this keeping the property mappings above?

Ideally, since i receive the JSON as a byte[], I would like to directly use the bytes (instead of encoding to a string - or if this has to be done at least encoding that doesn't add a lot of overhead).

I am OK to use attributes (for the JSON mapping) if this will make things simpler!

I am currently looking at MessagePackWriter to a raw buffer (working out memory management if I will need to use MemoryManager or just make my own byte[] arrays etc).

The idea will be to write the stream directly as i decode, and eliminate the Trade structure entirely:

private void GetTrade(ref Utf8JsonReader reader, IBufferWriter<byte> bufferWriter)
{
    var trade = new Trade
    {
        TradeConditions = new int[] { }
    };

    // working how to use writer out :)
    var writer = new MessagePackWriter(bufferWriter);

    while (Expect(ref reader, JsonTokenType.PropertyName, JsonTokenType.EndObject))
    {
        var property = reader.GetString();

        switch (property)
        {
            default:
                this.logger.LogError($"decoding {trade.GetType().Name} found unexpected property {property}, skipping");
                reader.Skip();
                break;
            case StreamFieldNames.Symbol:
                trade.Symbol = ExpectString(ref reader, StreamFieldNames.Symbol);
                break;
            case StreamFieldNames.ExchangeId:
                trade.ExchangeId = ExpectInt(ref reader, StreamFieldNames.ExchangeId);
                break;
            case StreamFieldNames.TradeId:
                trade.TradeId = ExpectString(ref reader, StreamFieldNames.TradeId);
                break;
            case StreamFieldNames.Tape:
                trade.Tape = ExpectInt(ref reader, StreamFieldNames.Tape);
                break;
            case StreamFieldNames.Price:
                trade.Price = ExpectDecimal(ref reader, StreamFieldNames.Price);
                break;
            case StreamFieldNames.TradeSize:
                trade.TradeSize = ExpectDecimal(ref reader, StreamFieldNames.TradeSize);
                break;
            case StreamFieldNames.TradeConditions:
                trade.TradeConditions = ExpectIntArray(ref reader, StreamFieldNames.TradeConditions);
                break;
            case StreamFieldNames.Timestamp:
                trade.Timestamp = ExpectUnixTimeMilliseconds(ref reader, StreamFieldNames.Timestamp);
                break;
        }
    }
    return trade;
}

donatasj87 · Answer 1 · 2020-05-05T10:32:16.277

If I understand correctly you are getting byte [] and want to get Trade structure without converting byte[] to string? In this case you can use Marshal library to cast unmanaged object to managed one:

    public T ByteArrayToStructure<T>(byte[] bytes) where T : struct
    {
        T result;
        GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
        try
        {
            result = (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
        }
        finally
        {
            handle.Free();
        }
        return result;
    }

You will need to include System.Runtime.InteropServices for this to work. And in this case byte[] will have to contain all fields in structure as declared, so you will not need to have any attributes. You can have more explanation and examples on codeproject website how does this work. While this problem would easily be solved in C or C++, here in C# we don't have pointers, which eases life for us, but also removes some functionality.

Thanks for taking hte time to reply. No i am looking to go straight from JSON to `StructureMap` schema, I am currently looking at `MessagePackWriter` and `IBufferWriter` — morleyc, May 05 '20 at 15:27
Do you mean you want to use Write command? https://github.com/neuecc/MessagePack-CSharp#messagepackwriter — donatasj87, May 06 '20 at 14:58

Direct json to StructureMap serialized byte[]

1 Answers1