Some unit tests runs indefinitely

Question

In a solution, I have 600 unit tests splitted in various libraries, and 450 of them are present in a single library. When running the tests on that specific library on my computer using Visual Studio (last version), everything goes well. But, on our build server, some tests never ends using the dotnet.exe test command.

dotnet.exe test C:\agent\_work\137\s\tests\XXX.Commercial.Infrastructure.Tests\XXX.Commercial.Infrastructure.Tests.csproj --logger trx --results-directory C:\agent\_work\_temp --configuration release -l "console;verbosity=detailed"

I tested the command on my local computer and everything goes well. I exectued the tests using Visual Studio on our build server directly and I could reproduce this issue on Visual Studio. I have to cancel the ones that never ends, and run them a second time to make them succeed.

Image of the Test Explorer on my local desktop

Image of the Test Explorer on the build server (test never end)

Image of Azure DevOps output after waiting 10 minutes after the last test and cancellation

The .csproj of the tests library where the issue occure

    <PropertyGroup>
        <TargetFramework>netcoreapp3.1</TargetFramework>
        <IsPackable>false</IsPackable>
        <ProjectGuid>{7BFDD481-8C96-41BD-9B8E-B7E1ECE9A6AC}</ProjectGuid>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="AutoFixture.AutoMoq" Version="4.11.0" />
        <PackageReference Include="AutoFixture.Xunit2" Version="4.11.0" />
        <PackageReference Include="coverlet.collector" Version="3.0.2">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <PackageReference Include="coverlet.msbuild" Version="3.0.2">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <PackageReference Include="FluentAssertions" Version="5.10.3" />
        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
        <PackageReference Include="Moq" Version="4.16.0" />
        <PackageReference Include="xunit" Version="2.4.1" />
        <PackageReference Include="xunit.runner.visualstudio" Version="2.4.1">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
    </ItemGroup>

On the internet, I found this but it's not related/doesn't solve my issue :

https://github.com/microsoft/vstest/issues/224

https://github.com/xunit/xunit/issues/1910

Edit

Here is a unit test that block for exemple :

[Fact]
public async Task SendSuccessReservationEmailConfirmationTest()
{
    Reservation reservation;
    using (var merchantContext = this._merchantContextFactory.Invoke())
    using (var reservationContext = this._reservationContextFactory.Invoke())
    {
        var merchant = await merchantContext.Merchants.FirstOrDefaultAsync();
        reservation = await reservationContext.Reservations
            .FirstOrDefaultAsync(x => x.MerchantId.Equals(merchant.Id));

        Assert.NotNull(reservation);
        Assert.NotNull(merchant);

        reservation.MerchantId = merchant.Id;
        reservationContext.SaveChanges();
    }

    await this._reservationNotificationService.SendReservationConfirmationAsync(reservation.Id);
}

Edit 2

Visual Studio : 16.9.5

DotNet CLI : 5.0.203

Any idea? Thank you for your help

Looks like they are the same tests that run indefinitely. I analyzed them and they don't really have anything in common ([Fact], [Theory], with/without [InlineData], [AutoMoqData], etc), but it's interesting. Running them sequentially works. Now I have a temporary solution. Thanks to https://stackoverflow.com/a/38558004/9563913 — Timothée Palumbo, May 17 '21 at 16:34
Yes, all of them, at different point. When calling the tested methods, calling the methods that create the SUT, or for method encapsulation. — Timothée Palumbo, May 17 '21 at 16:48
You might have a deadlock situation somewhere. They might lock the same resource . Can you post some of those never ending tests? — Peter Bons, May 17 '21 at 17:40
Yup, probably, but it's strange that it append now. 95% of the unit tests are more than 1.5 year old. I put an exemple in the description. Thank you for your help — Timothée Palumbo, May 17 '21 at 18:24
Hi @TimothéePalumbo, can you check the versions of Visual Studio and DotNet CLI between your local machine and your build server? If the versions are different between them. You can try to update the versions on your build server to be same as that on your local machine. Then try the tests again to see if they work. — Bright Ran-MSFT, May 18 '21 at 03:36
Hi. The versions are the same unfortunately. I'm investigating around the deadlock. Thanks — Timothée Palumbo, May 18 '21 at 09:08

score 1 · Accepted Answer · answered May 18 '21 at 09:24

I finally found the issue.

Using the "Parallel Stacks" tool (Debug -> Windows -> Parallel Stacks) of Visual Studio, I was able to locate the class/method where the deadlock occured.

Multiple tests where calling this methods :

/// <summary>
/// Builds this instance and starts the bus.
/// </summary>
/// <returns></returns>
public async Task<MassTransitHelper> BuildAsync()
{
    var bus = Bus.Factory.CreateUsingInMemory(configure =>
    {
        foreach (var (queue, configurator) in this._endpointConfigurators)
        {
            configure.ReceiveEndpoint(queue, configurator);
        }
    });

    await bus.StartAsync();

    return new MassTransitHelper(bus, new MessageBrokerSettings
    {
        MessageBrokerHostAddress = "loopback://localhost:5672/",
        MessageBrokerPassword = "guest",
        MessageBrokerUserName = "guest"
    });
}

However, multiple unit tests for a service where calling this method with a ".Result()" instead of an await, and they were all synchronous tests. Cleaning this by making them asynchronous and removing the ".Result()" solve the issue.

Thank you all for your help.

Hi @TimothéePalumbo, Thanks for sharing your experiences. I recommend that you can mark your answer as the solution of this topic. This may be very helpful to other people who are looking for a solution for the similar issue. Thanks. — Bright Ran-MSFT, May 26 '21 at 06:23
Hi @BrightRan-MSFT. Yep, sorry for the delay, I needed to wait 1 day and I forgot :-). Thank you for the reminder — Timothée Palumbo, May 26 '21 at 08:35

Some unit tests runs indefinitely

1 Answers1