Read a 4MB json file in the body of azure function by C#, and the Process hangs in body reading part

Question

dotnet version: 5.0.203 ide: JetBrain Rider

When i debug into the function, i can only see the threads in thread pool start and exit and not going into the breaking point.

All the reading and write was done by Task Async Await.

After that i tried MemoryStream, adding buffer size of reader to 40000+, and it still hangs.

Console logs: console logs

Reproduce Sample:

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.WebUtilities;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace XXXXXXXX.XXXXXX.Functions
{
    
    public static class TestFunction
    {
        [Function("TestFunction")]
        public static async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function,  "post")]
            HttpRequestData req,
            FunctionContext executionContext)
        {
            var logger = executionContext.GetLogger("TestFunction");
            logger.LogInformation("C# HTTP trigger function processed a request.");
            

            /* test01 */
            // var clDataJsonStr = await new HttpRequestStreamReader(req.Body, Encoding.UTF8, 4096).ReadToEndAsync();
            // var clDataJsonStr = await new HttpRequestStreamReader(req.Body, Encoding.UTF8, 4096).ReadToEndAsync();
            
            // req.Body.Seek(0, SeekOrigin.Begin);
            /* test02 */
            // var serializer = new JsonSerializer(); 
            // ClData clData;
            
            // using (var stream = new StreamReader(req.Body, Encoding.UTF8, false, 4096))
            // {
            //     using (JsonReader reader = new JsonTextReader(stream))
            //     {
            //         while (await reader.ReadAsync())
            //         {
            //             if (reader.TokenType != JsonToken.StartObject) continue;
            //             clData = serializer.Deserialize<ClData>(reader);
            //             logger.LogInformation(clData.ToString());
            //         }
            //     }
            // }
            
            /* test03 */
            // var stringReader = new StringReader(req.Body.ToString());
            // var str = await stringReader.ReadToEndAsync();
            
            // Console.WriteLine(str);

            /*test04*/
            var ms= new MemoryStream();
            await req.Body.CopyToAsync(ms);
            var jsonBytes = ms.ToArray();
            logger.LogInformation(jsonBytes.Length.ToString());
            
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

            await response.WriteStringAsync("Welcome to Azure Functions!");

            return response;
            
        }
    }
}

Test Json File: bycc.json

If the answer was helpful you may mark it accepted. – aepot Jul 08 '21 at 07:24 — aepot, Jul 08 '21 at 07:24
Does the below answer help? – Harshita Singh Aug 10 '21 at 10:45 — Harshita Singh, Aug 10 '21 at 10:45

Harshita Singh · Answer 1 · 2021-07-29T18:46:13.207

You can use StreamReader instead to read JSON body:

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
Class1[] data = JsonConvert.DeserializeObject<Class1[]>(requestBody);

This code works for your payload:

And, this is Class1 in above screenshot:

public class Class1
    {
        public string ReportDay { get; set; }
        public string Type { get; set; }
        public string ReportTypeDescription { get; set; }
        public string DoorName { get; set; }
        public string DoorCode { get; set; }
        public string Cc { get; set; }
        public float TotalSaleAmountIncludingVisitors { get; set; }
        public float TotalSaleAmountIncludingVisitorsLy { get; set; }
        public float TotalSaleAmountIncludingVisitorsEvol { get; set; }
        public float NewCustomerSaleAmount { get; set; }
        public float NewCustomerSaleAmountLy { get; set; }
        public float NewCustomerSaleAmountEvol { get; set; }
        public float OldCustomerSaleAmount { get; set; }
        public float OldCustomerSaleAmountLy { get; set; }
        public float OldCustomerSaleAmountEvol { get; set; }
        public float VisitorSaleAmount { get; set; }
        public float VisitorSaleAmountLy { get; set; }
        public float VisitorSaleAmountEvol { get; set; }
        public float CustomerSaleRatio { get; set; }
        public float TotalSaleQtyIncludingVisitor { get; set; }
        public float TotalSaleQtyIncludingVisitorLy { get; set; }
        public float TotalSaleQtyIncludingVisitorEvol { get; set; }
        public int NewCustomerSaleQty { get; set; }
        public int NewCustomerSaleQtyLy { get; set; }
        public float NewCustomerSaleQtyEvol { get; set; }
        public int OldCustomerSaleQty { get; set; }
        public int OldCustomerSaleQtyLy { get; set; }
        public float OldCustomerSaleQtyEvol { get; set; }
        public int VisitorSaleQty { get; set; }
        public int VisitorSaleQtyLy { get; set; }
        public float VisitorSaleQtyEvol { get; set; }
        public int TotalActiveCustomerQty { get; set; }
        public int TotalActiveCustomerQtyLy { get; set; }
        public float TotalActiveCustomerQtyEvol { get; set; }
        public int NewCustomerQty { get; set; }
        public int NewCustomerQtyLy { get; set; }
        public float NewCustomerQtyEvol { get; set; }
        public int OldCustomerQty { get; set; }
        public int OldCustomerQtyLy { get; set; }
        public float OldCustomerQtyEvol { get; set; }
        public float NewCustomerAchievingRate { get; set; }
        public float AusNotIncludingVisitor { get; set; }
        public float AusNotIncludingVisitorLy { get; set; }
        public float AusEvol { get; set; }
        public float NewCustomerAus { get; set; }
        public float NewCustomerAusLy { get; set; }
        public float NewCustomerAusEvol { get; set; }
        public float OldCustomerAus { get; set; }
        public float OldCustomerAusLy { get; set; }
        public float OldCustomerAusEvol { get; set; }
        public float IptNotIncludingVisitor { get; set; }
        public float IptNotIncludingVisitorLy { get; set; }
        public float IptNotIncludingVisitorEvol { get; set; }
        public float NewCustomerIpt { get; set; }
        public float NewCustomerIptLy { get; set; }
        public float NewCustomerIptEvol { get; set; }
        public float OldCustomerIpt { get; set; }
        public float OldCustomerIptLy { get; set; }
        public float OldCustomerIptEvol { get; set; }
        public int IptEqOneCount { get; set; }
        public int IptEqOneCountLy { get; set; }
        public float IptEqOneCountEvol { get; set; }
        public float PurchaseFreq { get; set; }
        public float PurchaseFreqLy { get; set; }
        public float PurchaseFreqEvol { get; set; }
        public float RecruitmentRate { get; set; }
        public float WechatBindingRate { get; set; }
        public float WechatBindingRateEvol { get; set; }
        public float CompanyWechatBindingRate { get; set; }
        public float CompanyWechatBindingRateEvol { get; set; }
        public float NewCustomerRepurchaseRatePerMonth { get; set; }
        public int NewCustomerRepurchaseQtyPerMonth { get; set; }
        public float NewCustomerRepurchaseAmountPerMonth { get; set; }
        public float NewCustomerRepurchaseAusPerMonth { get; set; }
        public float NewCustomerRepurchaseIptPerMonth { get; set; }
        public float NewCustomerRepurchaseRatePerThreeMonth { get; set; }
        public int NewCustomerRepurchaseQtyPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseAmountPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseAusPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseIptPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseRatePerSixMonth { get; set; }
        public int NewCustomerRepurchaseQtyPerSixMonth { get; set; }
        public float NewCustomerRepurchaseAmountPerSixMonth { get; set; }
        public float NewCustomerRepurchaseAusPerSixMonth { get; set; }
        public float NewCustomerRepurchaseIptPerSixMonth { get; set; }
        public int SkuServiceTimeCount { get; set; }
        public int SkuServicePeopleTimeCount { get; set; }
        public int SkuServicePeopleCount { get; set; }
        public int PurchasePeopleCountAfterService { get; set; }
        public float PurchaseRateAfterService { get; set; }
        public float PurchaseAmountOfPeopleCountAfterService { get; set; }
        public int PotentialCustomer { get; set; }
        public float PotentialCustomerInversionRate { get; set; }
        public float RepurchaseRate { get; set; }
        public float RetentionRate { get; set; }
        public float AnnualSpending { get; set; }
    }

Yes, you can. Check Joe's comment here: https://stackoverflow.com/questions/692263/system-io-stream-disposable-with-a-reader — Harshita Singh, Jul 07 '21 at 08:05
Mine azure function is isolated-worker(.NET 5+), and the req is actually HttpRequestData intead of HttpRequest of the old api. Would this be affected by the change of api? — Ronnie Yuan, Jul 14 '21 at 08:01
Yes, this will work for HttpRequestData: https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.functions.worker.http.httprequestdata?view=azure-dotnet also . Updated my code. Pls check — Harshita Singh, Jul 29 '21 at 18:46

Read a 4MB json file in the body of azure function by C#, and the Process hangs in body reading part

1 Answers1