22

From time to time, some requests on my website starts to hang on the RequestAcquireState state of the Session module. When that spiral begins all requests timeout and we need to restart the IIS on the affected server.

I investigated it a lot and the only conclusion I got is that somehow a deadlock is happening while the application tries to access user data stored in Session.

The only option I can think of to fix this issue is to either reduce or stop using Sessions in my application. This is definetely part of the plan, but it will take a while before we can complete that.

We run 6 machines with IIS 7.5, out of proc StateServer and server affinity on in our Load Balance.

Any hints on how to workaround this issue or fix it at all without having to remove Sessions entirely?

IIS Hanging Processes

tucaz
  • 6,524
  • 6
  • 37
  • 60
  • 1
    Have the same problem with my asp.net web forms 4.0 app & iis 7.5... any suggestion ? – RolandoCC May 15 '13 at 17:46
  • Is there by any case some page do a lot of work, eg, some process, or the download/upload of a big file ? – Aristos May 19 '13 at 14:12
  • No, most of the pages are read only. It's an e-commerce application. When the problem is not happening the average response times of all pages are between 70 and 80ms. – tucaz May 19 '13 at 18:11
  • Have you considered switching to SQL Server state mode? You can also eliminate server affinity in your load balancer and prevent users' sessions from getting dropped with a server meltdown. – Tombala May 24 '13 at 20:11
  • Another thing I would check is how bloated the session state is at time of failure. When you login to the server and look at task manager, find aspnet_state.exe, is it hogging memory, soaking CPU, etc? – Tombala May 24 '13 at 20:20
  • And one more thing... [Here's an article](http://bytes.com/topic/asp-net/answers/329298-aspnet_state-exe-internals-info-needed) that discusses how aspnet_state works behind the scenes and how things get "persisted" to there or how they might actually get held in the w3wp.exe process and not get flushed/GCed. Thought might be relevant reading for you. – Tombala May 24 '13 at 20:23
  • Next time the problem happens I will make sure to check that. Thanks. For now I'm reducing the amount of call to the Session by using a local cache to see if it improves. – tucaz May 26 '13 at 02:39

4 Answers4

7

Lock mechanism exist on both provider and session module (IIS Session Module). You can develop custom session module, but you still need provider without locking or You can develop custom provider without locking but you still need IIS session module and it is not so simple to implement at that level.

The Solution is UnlockedStateProvider [aka Unlocked]

Follow the white rabbit :P (Check the demo project, it explains everything.)

efaruk
  • 882
  • 9
  • 25
6

The answer is Hotfix Rollup 2828841 for .NET Framework 4.5 , here all the explanation:

http://forums.asp.net/t/1888889.aspx/2/10?Question+regarding+a+possible+bug+within+NET+4+5

and here the download link

It works for me on IIS 7.5 Windows Server 2008 rs x64 , asp.net web forms application with lot of ajax request.

RolandoCC
  • 868
  • 1
  • 14
  • 19
  • Thanks. I'm gonna try it and get back here. – tucaz May 17 '13 at 00:42
  • I didn't pay attention to the framework the hotfix applies to until I got it. I'm not using framework 4.5, but 4.0 so this hotfix does not apply. Thanks though. – tucaz May 17 '13 at 20:48
  • 1
    Actually the download link does not allow to download this hotfix. – Dmytro Laptin Dec 08 '15 at 18:18
  • Following is the discussion on a user forum where a person describes how he tried to contact Customer Support to get this hotfix. https://social.msdn.microsoft.com/Forums/vstudio/en-US/abc2ff86-c03b-4d4b-91b8-83006062f97b/download-hotfix-rollup-2828841?forum=msbuild – Dmytro Laptin Dec 08 '15 at 18:20
5

I just found out today that if you have a long running request (or in my case, an infinite loop), then all subsequent requests will be locked, because by default ASP.NET locks on session.

So if you have users with requests in RequestAcquireState, then check if there is a request in ExecuteRequestHandler that is locking the session, and thus preventing other requests from starting.

There is a discussion here on how to prevent locking on session. (Basically, create most of your pages as Session-Read-Only, and modify session as rarely as you can.)

Michael
  • 8,362
  • 6
  • 61
  • 88
jaraics
  • 4,239
  • 3
  • 30
  • 35
0

Is it possible those users have another long running request and the requests you see piling up are actually secondary requests? By default, ASP.NET will lock Session until a request is complete. If a second request comes in before the first one is complete, it will have to wait. If you are using MVC, you can change this behavior by adding an attribute to your controller.

[SessionState(SessionStateBehavior.ReadOnly)]

This makes Session read-only, removing the locking behavior allowing subsequent requests to be processed.

ctsears
  • 901
  • 1
  • 10
  • 11
  • Why would the first request hang in the first place? The issue is pretty much the same if that's the case. I'd still have hanging requests with no reason. – tucaz Apr 04 '14 at 04:39