Unit test ReSharper and NUnit give different results

Question

Using ReSharper 8.2 on local computer and NUnit 2.6.3 on build server found there were some test which passed in ReSharper and failed in NUnit. Installed NUnit locally and same results so it is not a difference between computers. Two of my colleagues ran the same tests and had same results so doesn't seem something messing my computer.

A simplified version of the tests:

[Test]
public void Test_UrlQueryString()
{
    var urlInput = "http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE";
    var uri = new Uri(urlInput);

    Assert.AreEqual(urlInput, uri.ToString());
}

[Test]
public void Test_Dot()
{
    var urlInput = "http://www.domain.com/page-with-dot.?p=google";
    var uri = new Uri(urlInput);

    Assert.AreEqual(urlInput, uri.ToString());
}

ReSharper output is all green. The output from NUnit:

Runtime Environment -
   OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
  CLR Version: 4.0.30319.18444 ( Net 4.5 )

ProcessModel: Default    DomainUsage: Single
Execution Runtime: net-4.5
...................F.F.........
Tests run: 29, Errors: 0, Failures: 2, Inconclusive: 0, Time: 0.576769973208475 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

Errors and Failures:

  1) Test Failure : Test.OrganicTest.Test_Dot
     Expected string length 45 but was 44. Strings differ at index 35.
  Expected: "http://www.domain.com/page-with-dot.?p=google"
  But was:  "http://www.domain.com/page-with-dot?p=google"
  ----------------------------------------------^

  2) Test Failure : Test.OrganicTest.Test_UrlQueryString
     Expected string length 87 but was 83. Strings differ at index 76.
  Expected: "...-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE"
  But was:  "...-with-querystring?url=https://www.domain2.com/page?p=PEPE"
  ----------------------------------------------------------------^

ReSharper seems to be using the same version of NUnit (Built-in NUnit 2.6.3)

ReSharper configuration

Does anyone knows what how to fix that? Is it a bug in ReSharper or NUnit?

score 10 · Accepted Answer · edited May 23 '17 at 12:07

Originally, it is a bug in .NET Framework. The URI scaping behaviour depends on environment (.NET Framework version and config). The bug was fixed in .NET Framework 4.5. For example, let's take the code

var uri1 = "http://www.domain.com/page-with-dot.?p=google";
var uri2 = "http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE";
Console.WriteLine(new Uri(uri1).ToString());
Console.WriteLine(new Uri(uri2).ToString());

Output in .NET Framework 3.5:

http://www.domain.com/page-with-dot?p=google
http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page?p=PEPE

Output in .NET Framework 4.5:

http://www.domain.com/page-with-dot.?p=google
http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE

Also, you can make some changes in escaping behaviour with your .config file. For example, there is an option for .NET 4.0 and slash escaping:

<configuration>
  <uri>
    <schemeSettings>
      <add name="http" 
           genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
    </schemeSettings>
  </uri>
</configuration>

I think, in your case, ReSharper and NUnit just use different configs. Although, it is really strange, that NUnit received old uri escaping results for .NET 4.5. Maybe NUnit use some microsoft libraries from .NET 4.0. Also, you can fail the test in ReSharper by manual set the Framework option to CLR4.0. Anyway, you should rewrite tests with environment independent logic.

Useful links:

This is exactly the right answer, but it's missing a key bit of information - .net 4.5 has a "quirks mode" (as explained in [this blog post](http://blogs.msdn.com/b/dotnet/archive/2012/10/17/net-framework-4-5-off-to-a-great-start.aspx) that gives the .net 4.0 behaviour when run in a .net 4.0 application. You'll need to have the latest ReSharper 8.2.3 to get a version that runs properly - no quirks mode. See [this YouTrack issue for the ReSharper bug](https://youtrack.jetbrains.com/issue/RSRP-412080) — citizenmatt, Nov 24 '14 at 10:26

Unit test ReSharper and NUnit give different results

1 Answers1