0

I am trying to serialize large object using below. But sometimes it throws out of memory exception.

Method 1:

            var settings = new JsonSerializerSettings { ContractResolver = BaseFirstContractResolver.Instance };
           string jsonString = JsonConvert.SerializeObject(listObj, Newtonsoft.Json.Formatting.Indented, settings);

So I have decided to serialize object to file like below,

Method 2:

            string path = System.Web.Configuration.WebConfigurationManager.AppSettings["jsonPath"].ToString();
            using (StreamWriter file = File.CreateText(path))
            {
                JsonSerializer serializer = new JsonSerializer();
                serializer.Serialize(file, listObj);
            }
        }

Also i am returning json data from file as HttpResponseMessage,

string path = System.Web.Configuration.WebConfigurationManager.AppSettings["jsonPath"].ToString();
                var stream = new System.IO.FileStream(path, System.IO.FileMode.Open);
                var response = this.Request.CreateResponse(HttpStatusCode.OK);
                response.Content = new StreamContent(stream);
                response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
                return response;

I am getting data fast in method 1 than method 2. Is there any performance difference between these two methods ?

vinoth kumar
  • 216
  • 4
  • 15
  • 3
    What did your profiler tell you? – Kenneth K. Apr 18 '18 at 13:38
  • 1) Obviously it's going to be faster to serialize and return JSON in memory rather than to a disk. But if you run of of memory you using memory you will have to do something else. 2) You're not using the `JsonSerializerSettings` in Method #2. – dbc Apr 18 '18 at 13:56
  • Take a look at [How to serialize a large JSON object directly to HttpResponseMessage stream?](https://stackoverflow.com/q/39355593) and [Using JSON.NET to serialize object into HttpClient's response stream](https://stackoverflow.com/q/25335897) which both suggest using `PushStreamContent`. – dbc Apr 18 '18 at 14:20
  • @dbc it is taking more time than methods i mentiioned – vinoth kumar Apr 18 '18 at 15:39
  • @vinothkumar - sorry that didn't work. Other suggestion would be to to return an `ObjectContent` as shown in, say, [Is it possible to change the MediaTypeFormatter to JSON for only one class?](https://stackoverflow.com/a/33169644) or [When to use HttpResponseMessage and Request.CreateResponse](https://stackoverflow.com/a/22066963), since it looks like [`BaseJsonMediaTypeFormatter.WriteToStream()`](https://github.com/aspnet/AspNetWebStack/blob/master/src/System.Net.Http.Formatting/Formatting/BaseJsonMediaTypeFormatter.cs#L411) writes directly to the `writeStream` rather than to a string. – dbc Apr 18 '18 at 17:02

1 Answers1

0

The first method is using memory while the second is writing to disk and then reading from disk again. The first method will always be faster, but it will consume more memory. The difference in performance will rely heavily on IO speeds of both memory and disk. I personally would avoid method 2 at all costs. I would implement paging on method 1 if the object was some type of collection. If I was forced to send the entire collection in one request then use a yeild return for chunks of the collection. That would create a chunked transfer encoding stream. If the object was indeed just a huge single object then some type of stream deserializer to again a chunked transfer encoding stream would be needed.

Alex Terry
  • 1,992
  • 1
  • 11
  • 17
  • JSON .NET is also stream deserializer. Even JsonConvert uses writing to stream internally. So you can use second method with MemoryStream, for example. Mostly the difference between SerivalizeObject:string and Serialize it that first will call stringWriter.ToString(); at the end – Evgeny Gorbovoy Apr 18 '18 at 14:37