0

I am struggling with the below exception. This exception is occurring when the API is receiving large JSON (1.21 GB).

 Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HMBV7L1S3NM9", Request id "0HMBV7L1S3NM9:00000002": An unhandled exception was thrown by the application.
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.GC.AllocateNewArray(IntPtr typeHandle, Int32 length, Boolean zeroingOptional)
   at System.GC.AllocateUninitializedArray[T](Int32 length)
   at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1.Rent(Int32 minimumLength)
   at System.Text.Json.JsonSerializer.ReadAsync[TValue](Stream utf8Json, Type returnType, JsonSerializerOptions options, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)

My JSON structure is:

{
  "base64": ""
}

Code is deployed on Kubernetes. please suggest me how can I fix this exception?

Lalit Rajput
  • 271
  • 2
  • 5
  • 23
  • 1
    Try to look at this [link](https://stackoverflow.com/questions/33480477/json-net-deserialize-out-of-memory-issue) – Den Sep 24 '21 at 11:35
  • I think it is wat you need :https://stackoverflow.com/questions/39955399/out-of-memory-exception-while-loading-large-json-file-from-disk – BASKA Sep 24 '21 at 11:37
  • 1
    What do you propose to do with 900 Mb `byte[]` array anyway? It's all very well using a stream, but the end result will still be a big array – Charlieface Sep 24 '21 at 11:40
  • @Charlieface I need to support big base64 in api request. – Lalit Rajput Sep 24 '21 at 12:20
  • I'm sure you'd like to, but if you don't have enough memory the only way you can do this is to stream it, hence my question: what do you want to do with it and can that be streamed? You cannot store it in memory, you don't have the space, and converting from base64 to `byte[]` is only going to save 300Mb – Charlieface Sep 24 '21 at 12:27
  • Actually, it is the default way of Microsoft binder. It is not written by me. I'm supposed I have to override the default binding. right? so I can write to disk instead of memory. Regarding the memory, the system has lots of available memory but .net core doesn't occupy large block of memory. – Lalit Rajput Sep 24 '21 at 12:36
  • Such data should not go through json, it should be for example passed as a file attachment in the body of http request. Then you could stream it directly to the storage. – Evk Sep 24 '21 at 12:40
  • @Evk what is the max size of request body supported by .net core? – Lalit Rajput Sep 24 '21 at 12:48

0 Answers0