1

I have a JSON structure something like this:

{
    Id: 1
    Data: [
        {
            X: 1, 
            Content: [...]
        }, 
        {
            X: 2, 
            Content: [...]
        }, 
    ]
}

Where Content can be very large byte arrays. The problem I have is that serialising and deserializing this structure using JSON.NET takes ups a large amount of memory on the LOH.

Is there some way I can WebAPI to stream the byte arrays to files?

The only other solution I can see is abandon JSON and use multipart/form-data but this seems kind of ugly, or is there some other way?

prb
  • 171
  • 1
  • 2
  • 10
  • Possible duplicate of [Incremental JSON Parsing in C#](https://stackoverflow.com/questions/9026508/incremental-json-parsing-in-c-sharp) – Drewness Feb 14 '18 at 00:15
  • gRPC/protobuf? https://www.strathweb.com/2013/02/asp-net-web-api-and-protocol-buffers/ – evilSnobu Feb 14 '18 at 00:17
  • 1
    There's a second option to shove those bytes into blob storage and send the URL inside `Content` (with a short-lived SAS if necessary). It depends on how dynamic that `Content` stuff really is. – evilSnobu Feb 14 '18 at 00:19
  • Incremental parsing looks like it would allow me to deserialize each of the Data elements one by one, but that would still bring the byte arrays into memory, which is what i want to avoid. – prb Feb 14 '18 at 02:42
  • I had a look at protobuf but the recommendation is not to use it for messages above 1MB or so, which is what I have. – prb Feb 14 '18 at 02:43
  • You're correct that Json.NET allows for incremental parsing of JSON files, but not for incremental parsing of **specific property values**. There's an enhancement open about this: [Add buffered reading of binary data to JsonTextReader #1462](https://github.com/JamesNK/Newtonsoft.Json/issues/1462). Also related (and unanswered): [Is there a XmlTextReader.ReadContentAsBase64 analog for JsonTextReader](https://stackoverflow.com/q/46365304) and [Pass Base64 encoded string using JsonTextReader value as new stream](https://stackoverflow.com/q/46670994). – dbc Feb 14 '18 at 03:24

0 Answers0