3

I have some code with unit tests that pass in a Debug build but fail in a Release build which is correct. However, the same tests pass in both Debug and Release mode when run using JetBrains dotCover.

To give a bit of background, here is the offending test code, just to give you in idea of why it's failing for a Release build - it's basically because of reduced stack information due to code optimization.

using System.Diagnostics;
using NUnit.Framework;

namespace DotCoverTest
{
    [TestFixture]
    public class TestLogger
    {
        [Test]
        public void GetCurrentClassLoggerReturnsLoggerWithOwningTypeName()
        {
            Assert.AreEqual(Logger.GetCurrentClassLogger(), GetType().Name);
        }
    }

    public class Logger
    {
        public static string GetCurrentClassLogger()
        {
            return new StackFrame(1, false).GetMethod().DeclaringType.Name;
        }
    }
}

EDIT: Any ideas how I can set up my build so that I get the same test results with or without a coverage tool ?

NOTE: This question was initially posted believing that it was a problem with TeamCity but it is not.

MickG
  • 31
  • 2

1 Answers1

2

I believe that the main issue here is Tail call optimization, Where the compiler collapses stack frames to boost performance. This happens only in Release mode.

dotCover (as other .net profilers) disables some of the CLR's optimizations, preventing compromised results - It would be hard to count method executions if they don't execute...

I can't tell you if NCover fails to overcome the optimizations, or solves this in another way, but i'm quite sure about dotCover.

seldary
  • 6,186
  • 4
  • 40
  • 55
  • I changed GetCurrentClassLogger() so that the line in question was not at the end and it still doesn't work as expected. It looks like you're right about the rest though. Do you know of a way to get around it ? – MickG Jan 31 '12 at 14:30
  • Changing the line order in the method won't help, since the entire frame (GetCurrentClassLogger) is using its parent frame. I don't see why you want to get around that... Either disable tail call optimization (~don't~) or don't use code that rely on the frame count. If you must rely on the stack, make the tests aware of that (it's hard to be precise without your real code of course). – seldary Jan 31 '12 at 16:06
  • I was just using this code as an example to illustrate the problem, I've had to remove it now anyway. Presumably there could be other things that could have the same effect after optimization than just StackFrame ? It would be nice if I could find a way to have Visual Studio and dotCover share the same build settings so that I can be sure that similar code doesn't slip under the radar in the future. Thanks for your help – MickG Feb 01 '12 at 09:02
  • I don't follow. dotCover does not "build". It is a .net profiler that changes the behaviour of the JIT compiler at runtime. Please explain what it is that you want to "get around" exactly - If you write code that depends on .Net optimizations, don't be surprised that it breaks on Release, and according to my answer, don't be surprised that dotCover bypasses optimizations... Write your tests accordingly (i.e you can run dotCover in a seperate run). – seldary Feb 01 '12 at 15:35