NSubstitute - TestFixture 1 causes AmbiguousArgumentsException in TestFixture 2

Question

I am writing C# unit tests using NUnit and NSubstitute. I am testing a class which will attempt to retrieve objects from a config provider implementing the following interface:

public interface IConfigProvider<T> {
    T GetConfig(int id);
    T GetConfig(string id);
}

The class being tested only uses the int version of GetConfig so in the SetUpFixture I do the following to set up a mocked config provider that will always return the same dummy object:

IConfigProvider<ConfigType> configProvider = Substitute.For<IConfigProvider<ConfigType>>();
configProvider.GetConfig(Arg.Any<int>()).Returns<ConfigType>(new ConfigType(/* args */);

This runs absolutely fine if that TestFixture is the only one being run. However, in a different TestFixture in the same assembly, I check for received calls like this:

connection.Received(1).SetCallbacks(Arg.Any<Action<Message>>(), Arg.Any<Action<long>>(), Arg.Any<Action<long, Exception>>());

If these Received tests run before the config provider tests, then the config tests fail in the SetUpFixture with an AmbiguousArgumentsException:

Here.Be.Namespace.ProfileManagerTests+Setup (TestFixtureSetUp):
SetUp : NSubstitute.Exceptions.AmbiguousArgumentsException : Cannot determine argument specifications to use.
Please use specifications for all arguments of the same type.
at NSubstitute.Core.Arguments.NonParamsArgumentSpecificationFactory.Create(Object argument, IParameterInfo parameterInfo, ISuppliedArgumentSpecifications suppliedArgumentSpecifications)
at System.Linq.Enumerable.<SelectIterator>d__7`2.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at NSubstitute.Core.Arguments.MixedArgumentSpecificationsFactory.Create(IList`1 argumentSpecs, Object[] arguments, IParameterInfo[] parameterInfos)
at NSubstitute.Core.Arguments.ArgumentSpecificationsFactory.Create(IList`1 argumentSpecs, Object[] arguments, IParameterInfo[] parameterInfos, MatchArgs matchArgs)
at NSubstitute.Core.CallSpecificationFactory.CreateFrom(ICall call, MatchArgs matchArgs)
at NSubstitute.Routing.Handlers.RecordCallSpecificationHandler.Handle(ICall call)
at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
at NSubstitute.Routing.Route.Handle(ICall call)
at NSubstitute.Proxies.CastleDynamicProxy.CastleForwardingInterceptor.Intercept(IInvocation invocation)
at Castle.DynamicProxy.AbstractInvocation.Proceed()
at Castle.Proxies.IConfigProvider`1Proxy.GetConfig(Int32 id)
at Here.Be.Namespace.ProfileManagerTests.Setup.DoSetup()

What's really confusing me is that I can observe this effect even between test runs - if I use the NUnit GUI to run the Received tests alone, and then run the config tests alone, the config tests will fail. If I then immediately run the config tests again, they will pass.

Things I've tried:

Adding configProvider.GetConfig(Arg.Any<string>()).Returns... as well, in case the overloading was the problem.
I've read the NSubstitute docs on argument matching, but I can't find a solution there. If it is a case of having to supply argument matchers for both the int and string versions of the method, I can't work out how to do that.

As it happens, the tests I'm using will only ever call the GetConfig method with values of 0 or 1, so I can just provide Returns specifications for those two values and not use matching at all, but I want to understand how to fix this more generally.

Are you using any arg matchers in the `new ConfigType(/* args */)` code? — David Tchepak, Nov 09 '14 at 01:32
No, I have an enum instance, a string, and an empty List - just placeholder arguments to create a just-good-enough object for the receiver to accept. — Tarrenam, Nov 10 '14 at 10:01

score 12 · Accepted Answer · answered Nov 11 '14 at 01:48

Ambiguous arguments is when NSubstitute compares the arguments to the call it is currently working with, to the stack of "argument matchers" it has (each time Arg.Blah is called, an the argument matcher is added to that stack), and it is unable to resolve which argument goes where.

Normally this is caused by having a call like blah(null, null), with a single argument matcher queued up, but it can also be caused by the stack getting out-of-sync due to an arg matcher being used outside of call configuration, or as an argument to a non-virtual method.

Version 1.8.0 (released after your question) includes slightly improved detection of the latter case, so that may be worth trying.

Other than that, I've had this problem a few times and have used the following (painful) approach.

run the test in isolation and ensure it passes
work out what test runs immediately proceeding (can usually guess, but test logs can help here), and run just those two tests. Confirm it fails.
Look for any calls to Arg.xyz that could queue up an argument matcher in either test. Make sure it is used as part of a call configuration. Sometimes working out which call is problematic can be done by commenting out lines or replacing arg matchers with other values.
Make sure there are no calls to non-virtual methods that are confusing NSubstitute.

Sometimes the problem may be due to a previous fixture, so you may need to workout the previous fixture and explore there as well. :(

"Make sure there are no calls to non-virtual methods that are confusing NSubstitute." That was the problem, I made various methods of `Connection` virtual, but `SetCallbacks` wasn't one of them. It is now, and the second fixture is now working fine :) — Tarrenam, Nov 12 '14 at 13:30

score 3 · Answer 2 · edited Feb 16 '18 at 14:13

I had similar errors that started when I switched Microsoft test runner to VSTest.Console (they did not happened when running under MSTest.exe).

As it was advised in David's answer, the errors were caused by calls to non-substituted methods with Arg.* parameters. Arg.Any were passed to actual code methods, that where called without Returns or Received relevant methods.

To scan my test library for such issues I used search with regular expression to find rows with Arg. but not Arg. following by Returns or preceded by Received

(?=^.*Arg.*$)(?=^((?!Arg.*\.Returns).)*$)^((?!\.Received\(.*Arg.).)*$

It's not bullet proof filter (e.g it doesn't exclude multi-line statements), but it helps to reduce number of calls to check.

Thank you Michael, this helps. A minor improvement I made was to add a period after the first Arg, like this `(?=^.*Arg\..*$)(?=^((?!Arg.*\.Returns).)*$)^((?!\.Received\(.*Arg.).)*$`. I also instructed Visual Studio's find tool to "Match case". — , Sep 22 '16 at 19:59

score 0 · Answer 3 · answered Dec 14 '16 at 16:24

0

Changing the order of my tests worked. Not a great answer, but worked - do try!

answered Dec 14 '16 at 16:24

arush436

1,748
20
20

score 0 · Answer 4 · answered Apr 14 '21 at 13:28

This is the regex I ended up using (in Rider, in my case, but it's a standard .Net regex). It handles multi-line statements pretty well. I did have some false negatives show up that had Returns() calls appended, but it was less than 1 percent of the total number of usages.

It does not handle When()/Do() or the WithAnyArgs variants, but those were by far the minority of the usages in my particular codebase.

^(?!\s*\.)[\w\s_\.]+\.(?!DidNotReceive\s*)(?!Received$\d*$\s*)[\w_]+$(((Arg.Any<.*?>\($)|(Arg.Is(<.*?>)?$.*$))[,\s]*)+\)(?!\s*\.Return)(?!\s*?\.Throw)

score 0 · Answer 5 · answered Apr 26 '23 at 13:43

I've solved the problem of finding mis-used Arg specs using an assembly level attribute, that verifies that before and after a test there are no queued up argument specs pending. It's for NUnit, but the concept should generalize to other unit test frameworks.

using System;
using System.Runtime.CompilerServices; 
using Tests;
using NSubstitute; // 4.2.2
using NSubstitute.Core;
using NUnit.Framework;
using NUnit.Framework.Interfaces;

// apply this ITestAction to all tests in the assembly
[assembly: VerifyNSubstituteUsage]

namespace Tests;

/// <summary>
/// This attribute can help if you experience <see cref="NSubstitute.Exceptions.AmbiguousArgumentsException"/>s.
/// It will ensure that no NSubstitute argument specifications are left in the queue, before or after a test.
/// This will happen if you pass <c>Arg.Any&lt;T&gt;()</c> (or other argument spec)
/// to an instance that is not generated with <see cref="Substitute"/><c>.</c><see cref="Substitute.For{T}"/>
/// </summary>
/// <remarks>
/// The <see cref="ITestAction.BeforeTest"/> and <see cref="ITestAction.AfterTest"/> will be run for every test and test fixture
/// </remarks>
[AttributeUsage(AttributeTargets.Assembly | AttributeTargets.Class | AttributeTargets.Method)]
public class VerifyNSubstituteUsageAttribute : Attribute, ITestAction
{
    public ActionTargets Targets => ActionTargets.Suite | ActionTargets.Test;
    public void BeforeTest(ITest test) => AssertNoQueuedArgumentSpecifications(test);
    public void AfterTest(ITest test) => AssertNoQueuedArgumentSpecifications(test);

    private static void AssertNoQueuedArgumentSpecifications(ITest test, [CallerMemberName] string member = null)
    {
        var specs = SubstitutionContext.Current.ThreadContext.DequeueAllArgumentSpecifications();
        if (specs.Count == 0) return;

        var message = $"{member}: Unused queued argument specifications: '{string.Join("', '", specs)}'.\n" +
                      $"Please check {test.FullName} test for usage of Arg.Is(...) or Arg.Any<T>() " +
                      $"with an instance not generated by Substitute.For<T>(...) ";

        Assert.Fail(message);
    }
}

NSubstitute - TestFixture 1 causes AmbiguousArgumentsException in TestFixture 2

5 Answers5

Linked