0

I just started exploring .NET Core Web App development this year and am still trying to understand a lot of things.

I'm currently testing the .NET Core Web App on IIS. An Application Pool with default settings has been assigned to it. Everything seems to be working as expected. Then, I execute a heavy process, in which the CPU usage of the IIS Worker Process will jump to 100% during execution, as monitored with the Task Manager. While it's executing, there are difficulties browsing the web app. Any requests sent have very slow responses.

The process

The process completely follows this sample How to: Write a simple Parallel.ForEach loop. It is executed by calling its API Controller. For our testing, it took 10 minutes to complete its execution. Therefore, the web app will be slow during that 10 minutes. If it helps to know, the server has 16 logical processors, which during execution, all of its jumped to 100%. CPU usage of the IIS Worker Process will then drop after execution finished. Below is the process' method.

private List<ModelX> CalculationProcess(IList<ModelY> data)
{
    var modelXs = new ConcurrentBag<ModelX>();

    Parallel.ForEach(data, datum =>
    {
        try
        {
            modelXs.Add(datum.GenerateModelX());
        }
        catch (Exception) 
        { 
            // ...
        }
    });

    return modelXs.ToList();
}

What has been done

I was under the assumption that it was just a matter of finding the right settings for the assigned Application Pool. So, I tested various settings, such as:

  • CPU Limit to 80;
  • CPU Action to KillW3wp, Throttle, and ThrottleUnderLoad;
  • Processor Affinity Enabled to true;
  • Maximum Worker Processes to 2.

While CPU usage of the IIS Worker Process did get limit to around 80%, the web app was still slow when the process is executing. I haven't tested setting the limit lower than 80%, and I'm unsure if it makes any real difference. I no longer know what other options to try.

enter image description here

How can I prevent the web app from becoming slow when a certain heavy process is executing on the back-end? I'm fairly new to both .NET Core Web App development and IIS.

azizanbps
  • 45
  • 6
  • Ideally you wouldn't be doing those sort of things in your app pool, especially if you want to have a responsive site. You could write a dedicated background process to do such things that lives outside of your application, or even outside of your webserver all together, maybe fire it off with a service bus, kafka, or rabbit MQ message. however it really depends on what you are trying to do, how fast you are trying to do it, and what your environment is – TheGeneral Aug 23 '21 at 03:19
  • @TheGeneral Thanks for your comment. Can I ask where should I go from here? By dedicated background process that lives outside of the application, would it work if I separate the API/Controllers from the web app, and assign it with a different Application Pool than the one for the web app? This was our initial plan, but ultimately scrapped. Process is only to calculate every single item in a list (can span hundred of thousands), I presumed the faster the better but so long it's within an hour is acceptable, and if I understood correctly the meaning of environment, it's Windows Server 2016. – azizanbps Aug 23 '21 at 04:33
  • Although it would be better to live outside of your production application pool, the problem is you only have so many cores and threads, if you use all your cpu doing cpu work everything else will suffer if it lives on the same system. If you cant move this to another PC, you could try limiting the degrees of parallelism, to leave cpu available for other things – TheGeneral Aug 23 '21 at 04:38
  • 1
    The root cause can be `Parallel.ForEach` in general, as in many case running multiple threads does always give you the best performance (thread creation and context switching are expensive operations). But about your specific case, optimization won't come easily without careful analysis and profiling. You didn't share the complete code base, so it is impossible for others to comment further. – Lex Li Aug 23 '21 at 05:16
  • We commonly recommend to use [DebugDiag](https://learn.microsoft.com/en-us/iis/troubleshoot/performance-issues/troubleshooting-high-cpu-in-an-iis-7x-application-pool) to troubleshot high cpu. But it's more likely that there's really need such a high cpu usage to finish the task, and as the suggestions offered above, it's better to move some tasks to another server, or we can recognize it as scale up or out. – Tiny Wang Aug 23 '21 at 06:44
  • @TheGeneral I forgot to mention this is still within a development environment. But your suggestion of limiting the degrees of parallelism is a good workaround, thanks! – azizanbps Aug 24 '21 at 03:45
  • @LexLi Thanks for the explanation. I apologize as I can't disclose any further codes, but the redacted code I provided really is the only thing being done there. As a workaround for my case, I updated the degrees of parallelism by half of processor count. That'll work for now. – azizanbps Aug 24 '21 at 03:51
  • @TinyWang, thanks for the tool recommendation! Hopefully we can try setting them up in separate server. But as a workaround, we're limiting the degrees of parallelism by half. It'll work for now. – azizanbps Aug 24 '21 at 03:57
  • @azizanbps That's great : ) – Tiny Wang Aug 24 '21 at 05:14

1 Answers1

0

It is very likely that Parallel.ForEach is consuming all the resources. You could limit this so that 1 process remains free for handling other requests by limiting the maximum number of cores that can be used.

See this question and answer for some more details: How can I limit Parallel.ForEach?

gjhommersom
  • 159
  • 1
  • 7
  • Yes, the process really did consume all the resources. I limit the parallel process to only use half of the total processor count, and the web app was able to serve requests better than before. Thanks for the suggestion! – azizanbps Aug 24 '21 at 04:04