2

I have a RESTful WCF service hosted on IIS 7.5. When some operation is called, it returns almost immediately, but starts a complex task, dealing with combinatorics and opening big files in memory. After several requests about 50% of memory is in use by application pool, although tasks have been completed. When does IIS pool reclaim memory? I tried to call GC.Collect(), but nothing happened. Is there any way to profile applications like this one? I tried several profilers, but they show only .NET classes, which IIS uses to process request itself.

Qué Padre
  • 2,005
  • 3
  • 24
  • 39
  • Could you show some code, how you create and Close the files in memory – Shiraz Bhaiji Jan 30 '14 at 22:31
  • @ShirazBhaiji Unfortunately, I can not post the code here, but I've tried to profile the same code by adding it to simple application and didn't find any leaks. – Qué Padre Jan 31 '14 at 07:55
  • Side note: not clear what you expect/complain about - if your operations are still in progress why do you expect objects to be released? Do you have reasons to believe that after all your "complex task" completed there is still large amount of managed memory used (don't look at task manager as it does not show % of free managed memory...) – Alexei Levenkov Feb 06 '14 at 07:25
  • @AlexeiLevenkov When all of my complex tasks completed task manager shows that 99% of server memory are in use by iis pool. I can not even start another web service... and the system is slow as hell... – Qué Padre Feb 06 '14 at 15:12
  • I think there is really nothing useful you can get from SO at this point - you need to grab minidump with single "complex task" when you think it is finished and analyze it to see if there are object you don't expect and like. Don't wait till process consumes all resources - looking through huge minidumps is not fun at all. Note that what you doing is generally frowned upon - running processes that are not handling requests inside IIS is questionable approach. – Alexei Levenkov Feb 06 '14 at 17:08

3 Answers3

3

The worker process itself will not release memory to the operating system on its own. You can set the process to recycle on a schedule - this restarts the process releasing memory without interfering with running requests.

You probably should not do that though - basically .net is holding on to the memory to avoid having to reallocate it for later requests. The memory is available for reuse within the WCF process, and if the memory is not used the OS will page it out and allow it be reused when other processes need it. See Answer to When is memory, allocated by .NET process, released back to Windows for more details.

Community
  • 1
  • 1
Rattle
  • 2,393
  • 2
  • 20
  • 25
  • Well, the request itself is processed extremely fast, as I said before. If I recycle a pool - the process which was started by request will die. – Qué Padre Jan 31 '14 at 07:53
  • What specifically is the issue, are you getting memory related errors, or just observing high memory usage in task manager? If you just see the process holding on to memory with no other errors then that is normal behaviour and you shouldn't try to change it. BTW, if you have a long running process that runs beyond the web request the worker process may not be the best place to run it, unless you have taken steps to stop IIS recycling the process automatically. – Rattle Jan 31 '14 at 11:05
  • Yup, the process which is started by a request is a long-running. The service consumes a lot of memory (I see it in a task manager) and I need to start another service right after, but there is no memory to do that. – Qué Padre Jan 31 '14 at 11:59
  • So do you get an error after handling a certain number of requests?What is the message logged? – Rattle Jan 31 '14 at 15:40
  • I don't have an error, I just see that 99% of system memory is used. – Qué Padre Feb 01 '14 at 15:57
  • @QuéPadre why can't you recycle as suggested? That is what everyone is doing it. You can recycle after every n number of requests. – Akash Kava Feb 09 '14 at 05:21
  • @AkashKava I don't know whether all the tasks were finished, so I can not recycle. For example, I sent 5 requests, they immediately return, but long-running tasks have been started. I can not do recycling at this point. – Qué Padre Feb 09 '14 at 11:01
  • @QuéPadre Recycle will only occur after it has finished last request successfully, while in recycle period, new process will start, new process will execute new request, old process will be destroyed only after it finishes all requests. Recycling is designed to handle this, and I am using this and have no problem. Alternatively, we created separate website with separate recycling model (1 recycle per request), and we internally call them from our main website (kind of web service proxy) – Akash Kava Feb 09 '14 at 11:10
  • @AkashKava >Recycle will only occur after it has finished last request successfully. That's it! But my request looks like "Start doing a long-running task". It returns almost immediately, so recycling could easily start while the complex task is still being processed. – Qué Padre Feb 10 '14 at 05:09
  • 2
    @QuéPadre That is the reason web server is not supposed to run background task, and azure was designed with queue & worker roles. You can create another process, let IIS recycle and your process must stop after one long task. Any task that takes more memory, must terminate itself after it is done. Windows will release memory when process terminates. – Akash Kava Feb 10 '14 at 05:31
3

Long running tasks don't typically suit web applications as they time out/hang the responsiveness of the website/API
Is it possible to configure a background task to run asynchronously of the IIS site? So you can push these slow tasks into a queue and process them in the background

I think the memory usage on the process is an issue but doesn't tell the whole story, what have you managed to profile so far? Do you have unclosed connections lingering? Are you creating instances of multiple classes that are not being disposed of effectively? I would look to profile the call execution plan more than the memory usage, as it may lead you more calls as to where items are being left

When you say 50% memory how much are we actually talking about in mb? IIS can be a little greedy/lazy when it doesn't need to give up RAM

finman
  • 545
  • 6
  • 16
  • well, 50% is 32 Gb here... =) I copied code to a console application and used a several profilers. So, basically this application does exactly the same, but without leaks. =) – Qué Padre Feb 04 '14 at 18:55
  • It may not be leaking, just not releasing what it doesn't need to until it does. Can you extract the methods used into a dll then call that from the service? It may help give a better insight to profile the application even if it's just to simulate some initial load on the service. This should show up things like unclosed connections, large objects on the heap not disposed of correctly and references left open. Once you have that you can look at the site itself to try and see if something isn't being disposed of correctly on that end, but IIS profiling is a bigger PITA than a DLL – finman Feb 07 '14 at 09:15
  • Can you also look in finer detail as to the memory usage if you're looking in task manager. You can add additional columns in order to see what the different types of memory allocation there https://www.dropbox.com/s/j5iwlkrjjexy9sb/memory%20settings.png (that is an example of an extended task manager view) – finman Feb 07 '14 at 09:18
  • Calling GC.Collect will pretty much guarantee nothing. It doesn't work like that http://www.worthalook.net/2014/01/give-back-memory/ . I'd also take a look at how the GC works to try and get a better understanding of where you can make the improvements yourself without trying to collect garbage http://blogs.telerik.com/justteam/posts/13-04-02/understanding-net-garbage-collection – finman Feb 07 '14 at 09:27
1

I had almost a similar issue, and i solved it by using Castle.Windsor as a IoC container, added the svc client class to container with Transient Scope, and finally, i have decorate the svc class with : [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)].

All other dependent bindings have beend added with Transient Livestyle, thus making them dependend by their instantiator. I`m not sure that this will help in your situation, given the fact that you use large files, but if every thing else fails, try to implement IDisposable on your most memory eaters classes, and check if Dispose is called when it should.

Hope that helps!

Florin V
  • 346
  • 2
  • 13
  • I`m not quite sure, but i think that the default WCF service factory returns the client svc class as singleton. – Florin V Feb 04 '14 at 15:58