2

I've run into this problem where NUnit tests aren't executed by Resharper's test runner. After a git bisect session I've isolated the commit that causes this, but I can't pin down why; most of the solutions I find refer to corrupt app.config files, but my commit only changed C# code. It only fails for one of my test projects - the unit tests - while other tests (integration and acceptance tests, also driven by NUnit) run fine.

So, I've tried to troubleshoot other ways, and following this guy's troubleshooting, I installed the NUnit Test Adapter for Visual Studio to try to run the tests with VS instead of R#.

Now, re-building the entire solution and checking the Test Output window, I see the following:

NUnit 1.2.0.0 discovering tests is started
Exception System.ArgumentNullException, Exception thrown discovering tests in D:\Code\ThisProject\src\MainWebApplication\bin\MainWebApplication.dll
Exception System.ArgumentNullException, Exception thrown discovering tests in D:\Code\ThisProject\src\UnitTests\bin\Debug\UnitTests.dll
NUnit 1.2.0.0 discovering test is finished

Hm... I did introduce a new method in this commit, that throws argument null exceptions. I wonder what happens if I comment out those checks?

NUnit 1.2.0.0 discovering tests is started
Exception System.ArgumentNullException, Exception thrown discovering tests in D:\Code\ThisProject\src\MainWebApplication\bin\MainWebApplication.dll
NUnit 1.2.0.0 discovering test is finished
A test with the same name 'UnitTests.SomeNamespace.SomeTestClass.SomeTestMethod(someparameter)' already exists. This test is not added to the test window.

Wait, what?

Removing an argument null check in my code, which is in a library assembly (i.e. in neither of the assemblies that initially failed, although both of them call into this method) made test discover perform (slightly) better. What is going on?

But it gets weirder still:

After having seen the above oddness, I resumed R# (which I had suspended as part of troubleshooting) and tried to run the tests again. They all run, and passed. Yes, I double checked and uncommented the null checks (changing nothing else) and I'm back to square one - Inconclusive: test wasn't run, for every single test in the assembly.

What is going on here?

I have no idea of how it could be relevant, but there are a couple of "exceptional" things about the particular method with the null checks, so I figure I can't leave them out of a question like this (where, to my knowledge, everything including Mickey Mouse could be relevant...):

  1. The method is often called with one of the parameters filled by a service location call (bad practice, I know, but it's a huge project with legacy code, and the calls are littered all over; there's no way to change that). If a call is made like that and the IoC container hasn't been setup, the argument will indeed be null.

  2. The method calls out to HttpContext.GetGlobalResourceObject, passing the arguments on (which is why I want a null check in the first place...), and we've configured a custom resource provider through web.config. I haven't changed any of this configuration, just moved the call to a different place.

Community
  • 1
  • 1
Tomas Aschan
  • 58,548
  • 56
  • 243
  • 402
  • It looks like the methods throwing `ArgumentNullException` are being called during test discovery. Are they static methods, or called from static classes/constructors? If you'd like to report the issue directly to JetBrains, where you could (confidentially) include source, you can do so here: https://youtrack.jetbrains.com/newIssue?project=RSRP&clearDraft=true&c= – citizenmatt Nov 04 '14 at 10:10
  • @citizenmatt: Thanks for the recommendation - [it's done](https://youtrack.jetbrains.com/issue/RSRP-427214). – Tomas Aschan Nov 04 '14 at 11:42

1 Answers1

2

With the help of R# support, I managed to squash this bug.

The root cause was that my static method, which threw the ArgumentNullException, was in one case called in the constructor of an attribute. When I refactored that attribute so that no exceptions could be caused during construction, the problem went away.

Without having confirmation, I guess that the problem with throwing exceptions from argument constructors is that NUnit instantiates the attributes during test discovery, and doesn't handle exceptions correctly. Thus, NUnit completely borks as soon as an attribute constructor throws anything, and this doesn't give any of the test runners a chance to let the user know what the problem is.

To find the offending code, it proved very useful to run VS with the following command:

devenv.exe /ReSharper.LogLevel Verbose /ReSharper.LogFile c:\path\to\logfile.txt

I then compiled my project and started a unit test run (in which all tests reported "Inconclusive; test wasn't run") and then exited VS again. Toward the end of the log file, there was a stack trace that told me where the exception was coming from.

Tomas Aschan
  • 58,548
  • 56
  • 243
  • 402