Unit Testing with Moq sometimes fails on ToListAsync()

Question

I'm fairly new to testing with moq and I'm having a strange issue (at least it seems strange to me), but I'm probably just not setting up the mock object correctly. I have a repository layer that uses EntityFrameworkCore to work with my DbContext. One particular function in the repository allows me to return a sorted list without exposing the Linq or EFCore functions to the service layer calling the function.

Say I have a Model class like this:

public class SomeClass {
    public string Foo { get; set; }
}

and I have a DbSet in my DbContext called someClasses. The three functions in my repository that I'm using to sort someClasses are these:

public async Task<List<SomeClass>> GetSomeClassesAsync(string orderBy = "", bool descending = false) {
    var returnVals = _context.someClasses.AsQueryable();

    returnVals = SortQueryableCollectionByProperty(returnVals, orderBy, descending);

    return await returnVals.ToListAsync();
}

private IQueryable<T> SortQueryableCollectionByProperty<T>(IQueryable<T> queryable, string propertyName, bool descending) where T : class {
    if (typeof(T).GetProperty(propertyName) != null) {
        if (descending) {
            queryable = queryable.OrderByDescending(q => GetPropertyValue(propertyName, q));
        } else {
            queryable = queryable.OrderBy(q => GetPropertyValue(propertyName, q));
        }
    }

    return queryable;
}

private object GetPropertyValue<T>(string propertyName, T obj) {
    return obj.GetType().GetProperty(propertyName).GetAccessors()[0].Invoke(obj, null);
}

So I have 2 unit tests for GetSomeClassesAsync(). The first unit test makes sure that the list returned is ordered by Foo, the second checks that an unordered list is returned when attempting to sort by Bar (a non-existent property). Here's how my tests are setup:

private Mock<DbContext> mockContext;
private MyRepository repo;

[TestInitialize]
public void InitializeTestData() {
    mockContext = new Mock<DbContext>();

    repo = new MyRepository(mockContext.Object);
}

[TestMethod]
public async Task GetSomeClassesAsync_returns_ordered_list() {
    var data = new List<SomeClass> {
        new SomeClass { Foo = "ZZZ" },
        new SomeClass { Foo = "AAA" },
        new SomeClass { Foo = "CCC" }
    };
    var mockSomeClassDbSet = DbSetMocking.CreateMockSet(new TestAsyncEnumerable<SomeClass>(data));
    mockContext.Setup(m => m.someClasses).Returns(mockSomeClassDbSet.Object);

    var sortedResults = await repo.GetSomeClassesAsync(nameof(SomeClass.Foo));

    Assert.AreEqual("AAA", sortedResults[0].Foo);
    Assert.AreEqual("CCC", sortedResults[1].Foo);
    Assert.AreEqual("ZZZ", sortedResults[2].Foo);
}

[TestMethod]
public async Task GetSomeClassesAsync_returns_unordered_list() {
    var data = new List<SomeClass> {
        new SomeClass { Foo = "ZZZ" },
        new SomeClass { Foo = "AAA" },
        new SomeClass { Foo = "CCC" }
    };
    var mockSomeClassDbSet = DbSetMocking.CreateMockSet(new TestAsyncEnumerable<SomeClass>(data));
    mockContext.Setup(m => m.someClasses).Returns(mockSomeClassDbSet.Object);

    var unsortedResults = await repo.GetSomeClassesAsync("Bar");

    Assert.AreEqual("ZZZ", unsortedResults[0].Foo);
    Assert.AreEqual("AAA", unsortedResults[1].Foo);
    Assert.AreEqual("CCC", unsortedResults[2].Foo);
}

DbSetMocking.CreateMockSet() was taken from here and TestAsyncEnumerable was taken from here

What I'm stumped on is the first test that returns an ordered list passes. Everything works fine. The second test fails and I get this error message:

System.NotImplementedException: The method or operation is not implemented.

This exception gets thrown when the code gets to ToListAsync(). What I don't get is why no error occurs when it goes through the work of sorting and then calling ToListAsync(), but when the sorting gets skipped and ToListAsync() is called, that exception is thrown. Am I not setting up my mock objects correctly?

Answer 1

The MosquitoBite's answer is the correct one. But for .Net 5 the ExecuteAsync method from the interface IAsyncQueryProvider changed a little:

TResult IAsyncQueryProvider.ExecuteAsync<TResult>(Expression expression, CancellationToken cancellationToken)
{
    return Execute<TResult>(expression);
}

And for IAsyncEnumerator the methods DisposeAsync and MoveNextAsync changed as well:

public ValueTask DisposeAsync()
{
    _inner.Dispose();
    return ValueTask.CompletedTask;
}

public ValueTask<bool> MoveNextAsync()
{
    return ValueTask.FromResult(_inner.MoveNext());
}

Answer 2

I would approach this completely differently than the other answers that have already been given. These answers are HEAVILY DISCOURAGED by Microsoft according to their Testing with EF Core documentation

Your goal is to test that your repository returns the right items based on the query you generate but that is done thanks to EF Core, a dependency. So you choose to mock it. But if you create all these fake classes to try and mock the context correctly, you might actually start testing your fake classes more than your actual code (in my opinion). What if EF Core changes the way their implementations of all those things you mock? Then your test might still succeed but your actual queries might change and work differently. Your unit test doesn't add a lot of value in that case.

If you want to do this perfectly, you would want to abstract that entire context away somehow so you would not even have to mess around with EF Core. But that is a lot of work and doesn't add a lot of value.

I believe creating quasi-integration tests is a lot more useful, which is the solution Microsoft also suggests.

I propose the following:

Read the overall testing with EF Core article and specifically the testing with EF In-Memory Database article from Microsoft. The solution they suggest is that you do use EF Core without mocking it, but instead of your actual database, you create a database in memory which uses sqlite under the hood. That way you can still test your code without having to create a lot of fake classes.

This is not a true unit test anymore, but if you read my previous links you understand that this is not what you want to be doing, anyway.

From this point on I am basically summarizing what microsoft says in their docs, but this approach has a few downsides:

• The database is sqlite. If you run a different database engine in prod, your tests and the queries that are generated by EF Core for the in memory database might not reflect the queries that generated at normal runtime.
• I have experienced some issues with the In memory database. Things like Include() sometimes did not want to work correctly, and some relationships between entities also acted funky sometimes. Maybe this is better nowadays.

As the docs already suggest, do some research and find out the limitations! I would suggest that if you have a lot of difficulty testing a specific repo call that just doesn't want to work, ask yourself if it is worth testing. If it is a simple GetAll() method, do you really need to test it that bad? Only add unit tests where it adds value!

I really hope this helps! Have a great day! :)

Answer 3

UPDATE AGAIN 2022-01-18

One should look at S. ten Brinke's answer.

My suggestion and Flavio's adding to it works and has been a practice suggested and documented by Microsoft. But time flies and EF Core with it and better approaches and features evolves.

Look at S. ten Brinke's answer and follow the links provided.

The best practice I think is to write your code so that your methods doesn't depend on database calls (or an EF abstaction of such). Move your data access code out into its own separate class(es) and give them the single responsibility to just work towards the database. Then use other methods than unit tests for testing data access code. Then you can put unit tests on the classes and methods that work with the data. The input to those methods should be easily faked and the database is out of the picture. In the data access code you should also have dealt with potential SqlExceptions.

PREVIOUS ANSWER

UPDATE

For dotnet 5 and up, look at this post AND Flavio's answer in this post, which gives you an update on the interfaces

The short answer is that there is nothing in your setup with Moq that causes the NotImplementedException to be thrown. It is the Ef Provider setup that needs to be set up to support async methods.

The looong answer is the following. :) I looked into what method to use when testing async methods that uses a EFCore context. It was not that obvious since the documentation on the setup used with Entity Framework 6 is really good but the documentation for EFCore is focused on the InMemoryProvider and SQLite-InMemory-Mode and does not include documentation for async testing and no hints that it is even supported. Or more accuratly, I did not find any.

So the solution I've found so far working with EFCore is the following:

1. Follow the EF6 async setup steps described in the MSDN Documentation This will get you the recipe for some wrapper classes and implementations on some interfaces (IDbAsyncQueryProvider,IDbAsyncEnumerable and IDbAsyncEnumerator) dot net core will not find those interfaces since they are named differently in Core so you will have to rename them.
2. Rename the interfaces to existing Core interfaces: The interfaces are located in Microsoft.EntityFrameworkCore.Query.Internal and are called IAsyncQueryProvider,IAsyncEnumerable and IAsyncEnumerator. So you just have to remove "Db" from the names of the interfaces.
3. Comment out everything except the constructors and private fields in the classes TestAsyncQueryProvider, TestAsyncEnumerable and TestAsyncEnumerator
4. Autoimplement the interfaces. Choose to "implement interface" where you have red squigglies on those classes and you will get the methods that these interfaces demands.
5. The implementations of the EF6 interfaces are similar to the EFCore interfaces. Just different names. So paste them in and adapt them.

Or, if you want to save time, copy and paste this code below. :) I just wanted to tell you how I got there, since this might not be a solution that will stand time. But until there's a standard solution (or at least until I find it) this seems to be the way to go.

public static class DbSetMockSetup
{
    public static Mock<DbSet<T>> SetupMockDbSet<T>(IEnumerable<T> dataToBeReturnedOnGet) where T : class
    {
        var mocks = dataToBeReturnedOnGet.AsQueryable();

        var mockSet = new Mock<DbSet<T>>();
        mockSet.As<IQueryable<T>>().Setup(m => m.Provider).Returns(new TestAsyncQueryProvider<T>(mocks.Provider));
        mockSet.As<IQueryable<T>>().Setup(m => m.Expression).Returns(mocks.Expression);
        mockSet.As<IQueryable<T>>().Setup(m => m.ElementType).Returns(mocks.ElementType);
        mockSet.As<IQueryable<T>>().Setup(m => m.GetEnumerator()).Returns(mocks.GetEnumerator());

        mockSet.As<IAsyncEnumerable<T>>()
            .Setup(x => x.GetEnumerator())
            .Returns(new TestAsyncEnumerator<T>(mocks.GetEnumerator()));

        return mockSet;
    }

}

internal class TestAsyncQueryProvider<TEntity> : IAsyncQueryProvider
{
    private readonly IQueryProvider _inner;

    internal TestAsyncQueryProvider(IQueryProvider inner)
    {
        _inner = inner;
    }

    public IQueryable CreateQuery(Expression expression)
    {
        return new TestAsyncEnumerable<TEntity>(expression);
    }

    public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
    {
        return new TestAsyncEnumerable<TElement>(expression);
    }

    public object Execute(Expression expression)
    {
        return _inner.Execute(expression);
    }

    public TResult Execute<TResult>(Expression expression)
    {
        return _inner.Execute<TResult>(expression);
    }

    public IAsyncEnumerable<TResult> ExecuteAsync<TResult>(Expression expression)
    {
        return new TestAsyncEnumerable<TResult>(expression);
    }

    public Task<TResult> ExecuteAsync<TResult>(Expression expression, CancellationToken cancellationToken)
    {
        return Task.FromResult(Execute<TResult>(expression));
    }
}

internal class TestAsyncEnumerable<T> : EnumerableQuery<T>, IAsyncEnumerable<T>, IQueryable<T>
{
    public TestAsyncEnumerable(IEnumerable<T> enumerable)
        : base(enumerable)
    { }

    public TestAsyncEnumerable(Expression expression)
        : base(expression)
    { }

    public IAsyncEnumerator<T> GetAsyncEnumerator()
    {
        return new TestAsyncEnumerator<T>(this.AsEnumerable().GetEnumerator());
    }

    IQueryProvider IQueryable.Provider
    {
        get { return new TestAsyncQueryProvider<T>(this); }
    }
    public IAsyncEnumerator<T> GetEnumerator()
    {
        return GetAsyncEnumerator();
    }
}

internal class TestAsyncEnumerator<T> : IAsyncEnumerator<T>
{
    private readonly IEnumerator<T> _inner;

    public TestAsyncEnumerator(IEnumerator<T> inner)
    {
        _inner = inner;
    }

    public void Dispose()
    {
        _inner.Dispose();
    }

    public T Current
    {
        get { return _inner.Current; }
    }
    public Task<bool> MoveNext(CancellationToken cancellationToken)
    {
        return Task.FromResult(_inner.MoveNext());
    }
}

And then you can use the setup like this ():

    public async Task Create_ReturnsModelWithANonEmptyListOfProducts()
    {
        var dbSetOfFoos = DbSetMockSetup.SetupMockDbSet(new List<Foo> { new Foo{ ... }});
        
        _context.Reset(); // _context is a Mock<MyContext>
        _context.Setup(db => db.Foos).Returns(dbSetOfFoos.Object);
        
        var sut = new ProductListViewModelFactory(_context.Object);
        var model = await sut.CreateAsync();
        // assert
        ...
    }

I hope this will help you to solve the issue. Good luck!