How can you compress a Json Action Result object in .Net Core?

Question

I am implementing a search for a list of people in MVC ASP.Net Core 2.1. Initially, no records are present and the user enters some filters and gets people returned. Currently, the max amount of records is 400,000 and so if they return all records we end up sending 80 MB of data back to the ajax request which we then loop through. This can scale up quite a bit as the final size of the database will be larger then it is currently.

We first get the data with LINQ and then we return an enumerable which we send back as a JSON Result object.

 [HttpGet]
 public async Task<IActionResult> GetPeopleList(FilterPersonListViewModel model)
 {
        _logger.LogInformation(
             message: GetLogDetail());

        //check if model is empty and if so send a empty collection back to initialise the table

        if (IsFilterModelValid(model))
        {
            ICollection<PersonEntityListViewModel> elvm = new List<PersonEntityListViewModel>();
            return Json(elvm);
        }
        bool isCanstat = _userService.IsInRole(User, Constants.Role.Canstat);
        var result = await _entityService.GetFilterPersonListViewModelAsync(model, isCanstat);
        var data = result.Model;

        return Json(data);       
 }

I then added the middleware to use Gzip compression in my Startup.cs file by adding the line ConfigureServices and Configure as it says to in https://learn.microsoft.com/en-us/aspnet/core/performance/response-compression?view=aspnetcore-2.1. The problem is it doesn't say how to send the response as Gzip. It says "Submit a request to the sample app with the Accept-Encoding: gzip header and observe that the response is compressed." but it doesn't say how I would enable that header in my response.

Do I have to do anything special to compress my data object when I send it back as Json? I found a few different articles relating to compressing it via GzipStream but then I can't send it back as an ActionResult it would seem?

"ajax": {
            "url": "/Person/GetPeopleList",
            "type": "GET",
            //"contentType": "application/json",
            "data": function (d) {
                setFilterData();
                Object.assign(d, filterData);
                return d;
            },
            "dataSrc": ""
        },

Are you sure you need to use compression? wouldn't increasing content length for request and response in config solve this problem? As Mentioned in these links http://rion.io/2013/04/28/handling-larger-json-string-values-in-net-and-avoiding-exceptions/ or https://stackoverflow.com/questions/5902540/increasing-maximum-response-size-from-asp-net-page-method — Hitesh Gaur, Oct 30 '18 at 19:36
Before applying compression to the response, the server needs to know if the client does accept that. So the problem is not on *how to set response headers* but on *how to set **request** headers*. You have to set the `Accept` header on request (ajax/jQuery side). Take a [look here](https://stackoverflow.com/questions/7988947/ajax-post-i-want-to-change-the-accept-encoding-header-value). — Anderson Matos, Oct 30 '18 at 20:22
@HiteshGaur 80 MB is pretty large to be sending when they issue a request so I'm trying to look into compression. Another thing is no compression is happening to any of the static files either. — Alex Demers, Oct 30 '18 at 21:22
@AndersonMatos I already looked into this and found this https://stackoverflow.com/questions/32463340/angular-gunzip-json-file-automatically-refused-to-set-unsafe-header-accept-en/32525490 basically it says the browser already automatically accepts gzip as a compression and sends it so I get the Refused to set unsafe headers error — Alex Demers, Oct 30 '18 at 21:22
I will recommend using either multi-part request/response, if there are files included in request and response, or use appropriate paging for records to send according the limit of maximum permissible data for your server. You can also stream data, if response includes media files. — Hitesh Gaur, Oct 30 '18 at 21:35
@AlexDemers you're right. I was just saying that it's a *request* issue, not a *response*, and that you should look into that. But you won't see much of it, since it will get compressed and uncompressed automatically for you. Problem is not about being zip-enabled but with the architecture, that should avoid that huge data chunck. Even if you enable 7Zip it will be way larger than the recommended 100Kb size limit. Consider chuncking or paging that data. — Anderson Matos, Oct 30 '18 at 21:41

score 0 · Accepted Answer · answered Nov 01 '18 at 18:33

0

Ye okay I needed to enabled EnableHttps = true to my middleware setup. Accidentally looked over that when I was reading the page. After adding that it ended up working as intended.

answered Nov 01 '18 at 18:33

Alex Demers

11
1
4

How can you compress a Json Action Result object in .Net Core?

1 Answers1