NSubstitute.Substitute.For() throws a FileNotFound exception but only using MSTest on Visual Studio

Question

We have problems with NSubstitute where it can't find certain DLLs. It seems the problem happens with mocking types from precompiled DLLs referencing types from other precompiled DLLs. So we have something like A.dll referencing types from B.dll which also references types from C.dll. There's an exception telling us that C.dll can't be found, even if it is there in the in the bin folder and properly referenced in the projects. All the versions also match. I managed to have some tests pass by removing references to the C dll. I also checkout the solutions for the precompiled and the DLLs versions are consistent. All the Nuggets are also up to date across the projects. All the builds architecture, release and language version are also consistent.

What makes this problem even weirder is that using VSTest.Console inside our continuous integration server doesn't reproduce the problem and I didn't see a decrease in the number of tests ran, it's only using VS Test Explorer. ReSharper's test explorer also does not have this problem. We saw this problem when upgrading to MSTest v2, the previous version worked.

Here's the stack trace (I truncated paths and namespaces) :

Result StackTrace:  
at System.Reflection.Emit.TypeBuilder.TermCreateClass(RuntimeModule module, Int32 tk, ObjectHandleOnStack type)
   at System.Reflection.Emit.TypeBuilder.CreateTypeNoLock()
   at System.Reflection.Emit.TypeBuilder.CreateTypeInfo()
   at Castle.DynamicProxy.Generators.Emitters.AbstractTypeEmitter.CreateType(TypeBuilder type)
   at Castle.DynamicProxy.Generators.Emitters.AbstractTypeEmitter.BuildType()
   at Castle.DynamicProxy.Generators.InterfaceProxyWithoutTargetGenerator.GenerateType(String typeName, Type proxyTargetType, Type[] interfaces, INamingScope namingScope)
   at Castle.DynamicProxy.Generators.InterfaceProxyWithTargetGenerator.<>c__DisplayClass6_0.<GenerateCode>b__0(String n, INamingScope s)
   at Castle.DynamicProxy.Generators.BaseProxyGenerator.ObtainProxyType(CacheKey cacheKey, Func`3 factory)
   at Castle.DynamicProxy.ProxyGenerator.CreateInterfaceProxyWithoutTarget(Type interfaceToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options, IInterceptor[] interceptors)
   at NSubstitute.Proxies.CastleDynamicProxy.CastleDynamicProxyFactory.GenerateProxy(ICallRouter callRouter, Type typeToProxy, Type[] additionalInterfaces, Object[] constructorArguments)
   at NSubstitute.Core.SubstituteFactory.Create(Type[] typesToProxy, Object[] constructorArguments, SubstituteConfig config)
   at NSubstitute.Substitute.For[T](Object[] constructorArguments)
   at OurTests.Fixtures.MockBuilder.Build() in C:\MockBuilder.cs:line 10
   at UnitTests.TestInitialize()

TestCleanup Stack Trace
   at Tests.TestCleanup() in 
Result Message: 
Initialization method Tests.TestInitialize threw exception. System.IO.FileNotFoundException: Could not load file or assembly 'AssemblyC, Version=1, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified..

TestCleanup method Tests.TestCleanup threw exception. System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object..

Note that we had problems when upgrading to MSTest V2 where it would run the debug version of our DLLs instead of the specified release ones, even if the path was absolute. I don't know if this is related.

This issue descibes the same kind of problem you are experiencing: https://social.msdn.microsoft.com/Forums/en-US/aa7f7567-faba-4589-b0ae-fe8d2c831c7a/tests-pass-in-ide-but-fail-with-mstestexe?forum=vststest — Bouke, Aug 28 '18 at 15:02
I had a similar issue, and I "fixed" it by not using MSTest and using NUnit instead. My root cause was that the object I was mocking referenced an older version of a nuget package than the unit-test project, and there was no binding redirect because the DLL in question had no public-key token (not sure if a binding redirect would have actually resolved this though) — RB., Aug 07 '19 at 10:53

TommyD · Answer 1 · 2018-08-28T20:35:41.697

After two days of mental health budget cuts, I finally found a fix. Or rather a patch since I'm still not 100% sure of the cause.

[ClassInitialize]
public static void ClassInitialize(TestContext testContext)
{
    AppDomain.CurrentDomain.AssemblyResolve += AssemblyResolveEventHandler;
}

private static Assembly AssemblyResolveEventHandler(object sender, ResolveEventArgs args)
{
    var dllName = args.Name.Split(',')[0] + ".dll";
    var assemblyPath = Path.Combine(BinDir, dllName);
    return Assembly.LoadFile(assemblyPath);
}

Note that this should probably be in the AssemblyInitialize method.

My best guess is that in the TestInitialize method, it doesn't make explicit calls to the DLLs so they do not get loaded at this point, and Castle (which NSubstitute uses) obviously does not know them yet, and probably try to load from the currently known assemblies. But when we run the tests from the continuous integration scripts, then other tests have ran before and so the DLL is already available (maybe the Global Assembly Cache?). As for why ReSharper's test runner worked, I do not know, my guess is that they have put a lot of developers candy into their products.

Also, credit where credit is due : https://stackoverflow.com/a/8967026/4602726

NSubstitute.Substitute.For() throws a FileNotFound exception but only using MSTest on Visual Studio

1 Answers1