1

My client has a desktop application that he uses to run his business. It runs against a local database. He wanted a web site to display some of this data.

I wrote an ASP.NET MVC5 web site, that includes an action that takes an HTTP POST containing some XML, and updates the web site's database based on the contents of the XML. His application makes multiple posts to the web site to upload the data. These posts are made on after the other, not simultaneously.

However, we have discovered that when he has a lot of data to upload, it will work fine for a while, then he gets a "Service unavailable" error. I checked with the hosting company, and they said that the pool log file was full of entries like this...

11/10/2015 4:15:03 AM --- a worker process serving application pool has requested a recycle because it reached its private bytes memory limit.

Anyone any ideas what I can do about this? The web site uses Entity Framework, which I realise is not the most memory-friendly way to do it, but to recode it all to access the database directly would be a huge job. I'm hoping there might be some easier way.

Would it help if he slowed down the requests? It occurred to me (possibly incorrectly as I'm not expert in these matters) that it might be that the garbage collector just isn't getting chance to free up the memory before more is requested. If so, would slowing down the rate of posts help?

As I said, I'm not being an expert in these areas, so please go easy on me!

Avrohom Yisroel
  • 8,555
  • 8
  • 50
  • 106

1 Answers1

2

Two reason may cause speed issue on web applications: thread starvation and huge data amount in transaction.

On the Web server, the .NET Framework maintains a pool of threads that are used to service ASP.NET requests. When a request arrives, a thread from the pool is dispatched to process that request. If the request is processed synchronously, the thread that processes the request is blocked while the request is being processed, and that thread cannot service another request.

This might not be a problem, because the thread pool can be made large enough to accommodate many blocked threads. However, the number of threads in the thread pool is limited. In large applications that process multiple simultaneous long-running requests, all available threads might be blocked. This condition is known as thread starvation. When this condition is reached, the Web server queues requests. If the request queue becomes full, the Web server rejects requests with an HTTP 503 status (Server Too Busy).

for "thread starvation" the best approach is using "Asynchronous Methods". refer here for more information.

About "huge data amount in transaction", you should check your code. May be you using too much data without need to all of them. For example you transfer all object which you may need just one properties of object. In this case use "projection"(refer here for an example).

Also you may use "lazy loading" or "eager loading" base on you scenarios. But please be noted that none of these are magic tool for every scenario. In some cases "lazy loading" improve performance and on others "eager loading" makes things faster. It depends to your deep understanding of these two terms and also your case of issue, your code and your design.

Community
  • 1
  • 1
Hadee
  • 1,392
  • 1
  • 14
  • 25
  • I see no evidence for thread amount issues (what makes you think that?) but the huge amount of data thing is probably spot on. – usr Nov 10 '15 at 20:52
  • If you have "thread starvation" issue, in most cases IIS uses queue to serve request and it takes time but not crash. So admin never see clear sign of that. It worth to check it and one thing to remember about that: use dbcontext very carefully. – Hadee Nov 10 '15 at 20:58
  • I just wonder why you think there might be many threads. I see nothing in the question that would suggest that. – usr Nov 10 '15 at 21:03
  • I just say "check it". If it isn't the case NP. – Hadee Nov 10 '15 at 21:05
  • Thanks for the detailed reply. To answer the 2nd question first, I doubt it's anything to with the amount of data, as each post is a very small piece of XML. I create EF objects based on the data, but each request is only going to require one or two objects, so it's unlikely that this is the problem. (cont...) – Avrohom Yisroel Nov 11 '15 at 14:05
  • As for the 1st point, running async is an idea, but if the server can't cope with so many requests in sync, isn't running them async going to give the same problem? At some point, the server isn't going to be able to cope with all the background processes, and will give the same error. Maybe it's my lack of understanding, but I'm not sure that running async would help. The server is going to have to cope with the same number of requests. Please can you comment on this? – Avrohom Yisroel Nov 11 '15 at 14:06
  • @AvrohomYisroel before you even consider going async you should confirm that you actually have many threads. – usr Nov 13 '15 at 17:39