byte[] + metadata to json

Question

I have some byte data that I want to attach some metadata to and serialize to some standard interoperable format. I figured I might be able to do this with json, in which case I'd like to have something like

{
  'metadata' : { /* ... */ },
  'data' : ???
}

What is a good way to do this?

I considered doing the data in base64 encoding, but then I'd come to something of the following:

class SerializationFormat {
  MyMetaData metadata {get; set;}
  string data {get; set;}
}

byte[] serialize(byte[] mydata, MyMetaData metadata){
  var obj = new SerializationFormat(){
    data = system.Convert.ToBase64String(mydata),
    metadata = metadata
  };
  //using json.net
  string jsonstring = JsonConvert.SerializeObject(obj);
  return System.Text.Encoding.UTF8.GetBytes(jsonstring);
}

I consider this rather silly; I'm taking byte data, encoding it in base64 which is guaranteed to be within 7 bit ASCII, encode it into a C# string (UTF-16), and decode it again into its original representation.

Is there a better way to do this?

Json.NET already converts a `byte []` property to a base 64 string, so you can skip that part. See [Serialization Guide](http://www.newtonsoft.com/json/help/html/SerializationGuide.htm). — dbc, Aug 31 '15 at 10:55
Also, you can stream directly to a [`MemoryStream`](https://msdn.microsoft.com/en-us/library/system.io.memorystream%28v=vs.110%29.aspx), as is shown in [Can Json.NET serialize / deserialize to / from a stream?](http://stackoverflow.com/questions/8157636/can-json-net-serialize-deserialize-to-from-a-stream/17788118#17788118), and skip the string representation. — dbc, Aug 31 '15 at 11:00
@dbc I'm only seeing it writing through a textwriter, first converting to UTF-16 base64, and then letting the textwriters encoder encode it back to UTF8, rather than writing to a `MemoryStream` — Martijn, Aug 31 '15 at 11:10
Something like [this](https://alexandrebrisebois.wordpress.com/2012/06/24/using-json-net-to-serialize-objects/), pass in a `MemoryStream`, then do [`memoryStream.ToArray()`](https://msdn.microsoft.com/en-us/library/system.io.memorystream.toarray(v=vs.110).aspx). — dbc, Aug 31 '15 at 11:14
That seems to be using a `TextWriter` and UTF-16 encoding for the base64 data as well — Martijn, Aug 31 '15 at 11:17
If it contains a way to write to a MemoryStream without converting the bytes to UTF-16 and then to UTF-8 then yes please, but this doesn't seem to be the case — Martijn, Aug 31 '15 at 11:33

score 0 · Answer 1 · answered Sep 02 '15 at 02:28

Here is an example of serializing an object containing byte data and metadata directly to a UTF-8 encoded stream. Json.Net handles the base-64 encoding of the byte data so you do not need to pre-process that. (Internally, Json.Net has its own Base64Encoder class that encodes the byte data to the JsonWriter in 76-character chunks, so you do not need to be concerned that the entire array is getting copied around in memory needlessly.) In this example I used a FileStream as the output, but you can replace that with whatever type of stream you want.

class Program
{
    static void Main(string[] args)
    {
        SerializationFormat obj = new SerializationFormat
        {
            metadata = new MyMetaData { foo = "xyz", bar = 12 },
            data = new byte[] { 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x57, 0x6f, 0x72, 0x6c, 0x64, 0x21 }
        };

        using (Stream stream = new FileStream(@"c:\temp\test.json", FileMode.Create))
        using (TextWriter textWriter = new StreamWriter(stream, Encoding.UTF8))
        using (JsonWriter jsonWriter = new JsonTextWriter(textWriter))
        {
            JsonSerializer serializer = new JsonSerializer();
            serializer.Serialize(jsonWriter, obj);
        }
    }
}

public class SerializationFormat
{
    public MyMetaData metadata { get; set; }
    public byte[] data { get; set; }
}

public class MyMetaData
{
    public string foo { get; set; }
    public int bar { get; set; }
}

Here is the output that gets written to the file:

{"metadata":{"foo":"xyz","bar":12},"data":"SGVsbG8gV29ybGQh"}

byte[] + metadata to json

1 Answers1