3

All,

Something's wrong with our test project throwing OOM exceptions, and I suspect the problem is us. We're holding references somehow and memory is never being released.

So, I'd like to run a memory profiler and see where this is happening.

Setting up a profiler tool is a simple matter of pointing the tool to nunit-console.exe and running our test project and taking a few snapshots.

Unfortunately, this doesn't work. I've tried both SciTech and demoed ANTS and both happily report that the NUnit console assembly/appdomain/whatever isn't growing. Great. The process in task manager has steadily grown to 450MB, but the memory profilers are reporting it hasn't grown at all.

Great.

I've googled, and I have seen some (tool-specific) simplistic instructions for how to deal with unit test projects. I've tried those things, and they didn't work. I've tried playing with NUnit's appdomain/assembly settings, thus far to no avail.

So.

Has anyone actually run a memory profiler against a test project (of any kind--we use NUnit, but I'd bet it's the same for any .NET automated testing framework) successfully? For any .NET test framework? If so, which instructions worked for you?

Charles
  • 50,943
  • 13
  • 104
  • 142
Peter Seale
  • 4,835
  • 4
  • 37
  • 45

4 Answers4

3

When we run into problems troubleshooting our application with unit tests (some of the libraries we use do not like running in a unit test environment) we simply create a console application that calls the test methods in the same order as the unit testing framework does. You then start the profiler on the console app.

You may not need to run all the tests to find out what the leak is.

We also encountered leaks in unmanaged memory that are not always visible from managed memory tracking tools. You need to specifically look for the unamanaged memory. These often result from missing Dispose() calls.

Vincent Hubert
  • 1,386
  • 9
  • 23
  • This is a last resort, but thanks for the answer. We use all the weird attributes and I don't want to reimplement all of NUnit, but we're getting desperate, so I may end up doing this. – Peter Seale Nov 16 '11 at 16:24
  • After more toughts, I am not sure the problem lies in using it with NUnit. Did you try voluntarilty inserting a leak in your tests to see if the profiler can detect those? (just add stuff to a collection between snapshots) It is possible that the leaks themselves are in a place hard to detect by the tools and that the difficulty is inherent to the leak itself, not the tool. IIRC, the last tab in the profiler give stats relative to the whole process. I once had a leak related to ADO .net where a query would leak a few K in unmanaged memory after each call. – Vincent Hubert Nov 18 '11 at 15:01
  • No, it's clearly not showing any data from the tests themselves. I followed your advice and created Peter's NUnit Console project and ran a few of our tests yesterday, and I think I've found the leak. I'm just curious why it's just me that profiles test projects. #probablydoingsomethingIshouldnt – Peter Seale Nov 18 '11 at 20:31
0

I have used the latest version of RedGate ANTS (v9.5.0.853) and it correctly (and easily) profiled my .NET libraries via NUnit (v2.6.2).

You need to set the EXE to launch as the NUnit exe... And I targeted a particular feature in my DLL ("Portal.Interface.dll").

I created new profile session;

  • Path to .NET executable: C:\Program Files (x86)\NUnit 2.6.2\bin\nunit-x86.exe
  • Command Line args: Portal.Interface.dll /fixture:Portal.Interface.Tests.TestSerializingSpeed
  • Working Directory:

Then I ran it with highest profiling method.

It produced the leak results I was expecting...

James Joyce
  • 1,694
  • 18
  • 20
0

You should try JetBrains MemoryProfiler(I'm using version 3.5) It works great on NUnit(GUI version)!

6opuc
  • 1,176
  • 3
  • 13
  • 21
-1

Have you ensured that you've considered all types of allocations, including unmanaged memory?

By the way, OutOfMemoryException isn't necessarily thrown when there's no memory free at all. It's sometimes thrown by certain classes that want to allocate large contiguous blocks of memory, for example StringBuilder. XmlDocument.SelectNodes throws it on occasion.

Unfortunately it's something you can't do anything about. If the application is busy, the garbage collector is within its rights to sit tight and refuse to free memory, and the likelihood of failing to find a contiguous block to allocate grows.

edit #1

I may have been misremembering (blame the beer) but the situation you're observing can also be explained by the fact that the garbage collector reports that it has released objects, but the OS' memory manager hasn't got around to fully freeing the memory formerly occupied. It's even more entitled to do what it feels like than the .NET garbage collector is. The best solution to this sort of problem is to design around anything that tries to allocate massive value types, most conspicuously System.String.

Tom W
  • 5,108
  • 4
  • 30
  • 52
  • Downvoters - please have the decency to explain why. I have personally experienced similar problems to the ones described, and the root cause is what I've mentioned in my post. – Tom W Nov 16 '11 at 16:34
  • Downvoted because it doesn't help me with my problem today. It may be the answer to someone else's question, but it doesn't tell me how to run a profiler against a test project. – Peter Seale Nov 16 '11 at 17:05
  • I thought I explained myself well enough - what I'm saying is that I suspect you won't find what you're looking for by profiling the test project. The **question** is how to profile a test project, but the **problem** that you open the question with is the OutOfMemoryException. If it's the issue I've described, profiling won't find it. – Tom W Nov 17 '11 at 14:28