37

I am seeing some very strange behavior for NUnit tests in VS2019, where the same solution works fine in VS2017. There are several NUnit test projects in my soultion.

In VS2017 with the NUnit Runner extension installed, I can see all of my tests in the Test Explorer window and the "run all" button will work and run all tests. Some developers in my organization use Resharper instead of the NUnit extension and this works too.

I have stopped using Resharper because as VS introduces more features, Resharper has made it so slow that VS is unusable.

In VS2019 the Test Explorer window will show all of my unit tests (even without the NUnit extension installed). If I click "run all" it will not run any tests and the Output window will say 0 tests discovered. Coworkers have said that Resharper will run all tests without issue. If I right click a single test project and run just those tests, some projects will run tests, but not all.

For some projects I have tried installing the NUnit3TestAdapater nuget package and this will let VS2019 run that project's test if only that project is selected. This does not work for all projects and it still does not work for "run all".

Does anyone know what might be causing this and what could fix it? I have updated to the latest version of NUnit (3.12) and latest TestAdapter (3.16) for all of these projects.

This has been a real pain for about a year, because I need to keep VS2017 and 2019 both installed, and I need to train new developers on how to work around this weird issue.

JamesFaix
  • 8,050
  • 9
  • 37
  • 73
  • 1
    The Visual Studio team dropped support for extension based test adapters in VS2019 - see here: https://github.com/nunit/nunit3-vs-adapter/issues/576. Your issue with 'run all' not working isn't familiar, have you tried removing the extension, and just having the adapter NuGet package installed in the relevant projects? If so, I would suggest filing an issue on the nunit3-vs-adapter GitHub repo. – Chris Apr 14 '20 at 22:28

7 Answers7

40

I stumbled upon the same problem. It seems to me that I have to install more and more with each release. In my case (Visual Studio 2019 community edition, version 16.6.1), I now also had to obtain the Microsoft.NET.Test.Sdk via NuGet.

So in the end, I have installed three packages in my test project:

  • NUnit (3.12.0)
  • NUnit3TestAdapter (3.17.0)
  • Microsoft.NET.Test.Sdk (16.7.1)

Turning the automated test discovery on or off did not change anything for me though.

Hope this helps anyone.

Thomas Schütz
  • 451
  • 4
  • 2
  • 1
    The test discovery option only seems to matter with parameterized tests. I think the Microsoft.NET.Test.Sdk package has been a requirement for .NET Core projects for a while, but not on .NET Framework. Which version are you using? – JamesFaix Oct 19 '20 at 16:49
  • 2
    I have exactly same setting, and installing Microsoft.NET.Test.Sdk fixed my problems. But, there is no word about Ms.NET.Test.SDK in instructions of NUnit. Thanks anyway – Kamil Oct 21 '20 at 11:48
  • This worked for me running VS 2019 community, thanks! – Luke Belbina Nov 20 '20 at 22:29
  • Have followed all the steps in this question and no joy, tests are not getting discovered VS 2019 community Edition –  Jan 04 '21 at 19:49
  • 2
    For me only added NUnit3TestAdapter (NUnit already installed). Didn't need the MS Sdk. – Eric Carlson May 12 '21 at 23:56
  • In My case it was `Microsoft.NET.Test.Sdk` and yes there is no mention about the requirement – Supun De Silva Jul 11 '22 at 13:16
  • I only needed to add NUnit3TestAdapter like @EricCarlson. Runtime error with the latest NUnit3TestAdapter 4.4.2 then showed an old initialize method needed adjustments due to invalid method parameters where nUnit [SetUp] attribute was set. – InquisitionX Mar 02 '23 at 20:05
32

The solution turned out to be a combination of two things.

  1. On the top menu, navigating to Test > Options and disabling "Discover tests in real time from C# and Visual Basic .NET source files". This option appears to be incompatible with parameterized tests. The problem is that parameterized tests do not "exist" in source code but are generated by the test adapter at runtime. This also fixed an issue I've seen where the "base" test of the parameterized test shows up in the Test Explorer as a not-run test, even though only the specific test cases are real tests. (In my opinion, this should not be enabled by default because parameterized tests are extremely useful, whereas seeing new tests in the Test Explorer without compiling is a trivial convenience since you must compile to run them anyway.)

  2. Using the NuGet package for the test adapter instead of using the VSIX extension. It seems that for this piece, all that is required is that at least one project in your solution references this. If at least one project references it, all test projects can be run. (This makes sense to me, as it is more compatible with build tools outside of Visual Studio.)

Hooray for breaking changes!

JamesFaix
  • 8,050
  • 9
  • 37
  • 73
  • 2
    re: `#2` Which NuGet package(s)? There are so many to pick from. – Jesse Chisholm Oct 31 '20 at 19:49
  • 2
    @JesseChisholm likely `NUnit3TestAdapter` – emragins Nov 16 '20 at 20:43
  • 1
    1. Disabling "Discover tests in real time from C# and Visual Basic .NET source files" did some improvement (Visual Studio 2019, v16.8.2; xunit v2.4.1; xunit.runner.visualstudio v2.4.3 ) but its still much slower than Resharpers test-explorer – JimiSweden Jan 15 '21 at 05:58
  • 1
    Agreed Resharper is still ahead there, but since VS2017 I havent been able to run Resharper because its too slow in all other regards and anyway VS has caught up to a lot of its features. – JamesFaix Jan 15 '21 at 13:53
  • Wow, never expected this to be the issue. Thanks a lot – 3lvinaz Sep 21 '21 at 06:47
  • For the #2, I am still confused how to use NuGet package for the test adapter. What to do to change the test adapter. @JamesFaix, could you show screenshot? – Yang You Oct 19 '21 at 03:39
7

I had the same issue and solved by checking the logging.

In Tools->Options->Test->General setting the Logging Level to Diagnostic. This will produce additional output in the Tests Output Pane of the output window.

In my case the issue was linked to missing version of ".NET Core 3.1 Desktop Runtime" You can download the last version in https://dotnet.microsoft.com/download/dotnet/thank-you/runtime-desktop-3.1.14-windows-x86-installer

Yassine
  • 331
  • 1
  • 3
  • 9
5

In my case I had to install also the Microsoft.NET.Test.Sdk nuget pack.

Tiger Galo
  • 289
  • 4
  • 15
5

After none of the above worked for me.

I just installed xunit.runner.visualstudio nuget and everything is working fine now.

Dongolo Jeno
  • 414
  • 6
  • 8
0

Similar experience in VS22 - Updated my NUnit3TestAdapter to 4.4.2

Mike
  • 629
  • 5
  • 18
0

In my case, I had recently created a new test class using VS 2019, which sets the test class to internal by default. Removing internal and setting the test class to public solved it.

Vincent Karuri
  • 880
  • 6
  • 4