2

I was trying to implement a subclass of ActionResult that will stream big JSON objects from a REST API, I found this solution on stack overflow but it seems like it's an implementation for asp.net MVC.

public class JsonStreamingResult : ActionResult
{
    private IEnumerable itemsToSerialize;

    public JsonStreamingResult(IEnumerable itemsToSerialize)
    {
        this.itemsToSerialize = itemsToSerialize;
    }

    public override void ExecuteResult(ActionContext context)
    {
        var response = context.HttpContext.Response;
        response.ContentType = "application/json";
        response.ContentEncoding = Encoding.UTF8;

        JsonSerializer serializer = new JsonSerializer();

        using (StreamWriter sw = new StreamWriter(response.OutputStream))
        using (JsonTextWriter writer = new JsonTextWriter(sw))
        {
            writer.WriteStartArray();
            foreach (object item in itemsToSerialize)
            {
                JObject obj = JObject.FromObject(item, serializer);
                obj.WriteTo(writer);
                writer.Flush();
            }
            writer.WriteEndArray();
        }
    }
}

But when I was in the process of porting this to asp.net core MVC I found that the response class does not have ContentEncoding and OutputStream properties.

Please, can anyone provide the required changes to port this class to asp.net core?

Thanks in advance.

  • 2
    Are you sure you need that in the first place? When you call `return Json(someObject)` you get back a [JsonResult](https://github.com/aspnet/AspNetCore/blob/master/src/Mvc/Mvc.Core/src/JsonResult.cs) object that eventually [uses a JsonWriter](https://github.com/aspnet/AspNetCore/blob/master/src/Mvc/Mvc.NewtonsoftJson/src/JsonResultExecutor.cs#L108) to write to the response output stream. Have you tried returning your data in the normal way? – Panagiotis Kanavos Apr 11 '19 at 11:36
  • @PanagiotisKanavos Please see this question, I have the same issue : https://stackoverflow.com/questions/26269438/streaming-large-list-of-data-as-json-format-using-json-net – Ondama papaoutai Apr 11 '19 at 11:43
  • That question isn't about ASP.NET Core and the problem there was *caused* by the attempt to manually serialize the data. Do you have an actual problem? Have you tried using `return Json()`? I posted links to the source code that show that ASP.NET Core *already* writes to the output stream – Panagiotis Kanavos Apr 11 '19 at 11:49
  • The reason people wrote their own JsonResult classes in ASP.NET MVC was that it used the obsolete JavaScriptSerializer. ASP.NET *Web API* replaced this with Json.NET and a different API. ASP.NET *Core* MVC evolved from Web API so it doesn't need such workarounds – Panagiotis Kanavos Apr 11 '19 at 11:52
  • @PanagiotisKanavos the problem is when I send back the result a Json array of objects that contains like 30 000 objects it takes like a minute to load it . Is there a way to optimize this ? I'm really facing the same problem which is perfromance issue when streaming the data from the server to the client. – Ondama papaoutai Apr 11 '19 at 11:53
  • 2
    How do you know? Did you time the entire action execution or the serialization parts? If the connection is slow, retrieving the data will take a lot of time. If *loading* the data is slow, the entire operation will take a lot of time no matter how the data is serialized. You'll have to profile your code, or at least debug it and see what's actually taking that long. – Panagiotis Kanavos Apr 11 '19 at 12:00
  • You should consider adding StackOverflow's [MiniProfiler](https://github.com/MiniProfiler/dotnet) to your pages to profile and check how long each stage in an actin really takes. It can show separate timings for SQL, EF, rendering, network operations while loading a page – Panagiotis Kanavos Apr 11 '19 at 12:02
  • @PanagiotisKanavos It is the serialisation process. The default asp.net core serializer Json.Net takes 30 seconds to serialise a list of 20 0000 object. – Ondama papaoutai Apr 11 '19 at 14:08
  • Can you please give me any advice to speed up this process. – Ondama papaoutai Apr 11 '19 at 14:08
  • 1
    How about stop returning so much data at once? JSON is pretty verbose: not as verbose as XML, but that's not really saying much. Returning 20,000 JSON objects is going to take a while - no matter what you do. The correct approach here is to *page* the results. Return something like 100 at a time, and then allow the client to request the next "page" of 100 results when they actually want it. This is far more efficient anyways, as it would be rare that all 20,000 are *actually* needed. – Chris Pratt Apr 11 '19 at 17:36
  • Aside from that, you'd need to use a more efficient message type, such as MessagePack (essentially binary JSON) or Protobuf. However, support for neither is built in, so you'd need to find a third-party library or write some formatters yourself. – Chris Pratt Apr 11 '19 at 17:38
  • @ChrisPratt Thank you for your answer, it's just that the client needs to get it at once, I'm aware that paging is the right and the way to go, but please is there any workaround to make this work ? – Ondama papaoutai Apr 11 '19 at 17:47
  • The workaround would be to use a more efficient message format, such as MessagePack or Protobuf, as I've already stated. Other than that, you're out of luck. – Chris Pratt Apr 11 '19 at 18:07

1 Answers1

3

OutputStream - in ASP.NET Core HttpResponse contains Body property to which you can write a response. ContentEncoding - set encoding for StreamWriter since you manually write result to the response stream. In ASP.NET MVC HttpResponse.ContentEncoding were only used when you called HttpResponse.Write methods.

public class JsonStreamingResult : ActionResult
{
    private IEnumerable itemsToSerialize;

    public JsonStreamingResult(IEnumerable itemsToSerialize)
    {
        this.itemsToSerialize = itemsToSerialize;
    }

    public override void ExecuteResult(ActionContext context)
    {
        var response = context.HttpContext.Response;
        response.ContentType = "application/json";

        JsonSerializer serializer = new JsonSerializer();

        using (StreamWriter sw = new StreamWriter(response.Body, Encoding.UTF8))
        using (JsonTextWriter writer = new JsonTextWriter(sw))
        {
            writer.WriteStartArray();
            foreach (object item in itemsToSerialize)
            {
                JObject obj = JObject.FromObject(item, serializer);
                obj.WriteTo(writer);
                writer.Flush();
            }
            writer.WriteEndArray();
        }
    }
}

Update

According to source code, JsonResultExecutor internally does exactly what I've described, only difference is it parses encoding from ContentType.

Alexander
  • 9,104
  • 1
  • 17
  • 41