No tests discovered using xUnit on ASP.Net Core 1.0 project with build on Visual Studio Team Services Hosted agent

Question

I have a problem where my xUnit tests in my ASP.Net Core 1.0 application are not discovered when building on Visual Studio Team Services.

The test project is being built successfully, but no test results are given. I find the following messages regarding the Visual Studio Test step in the log:

******************************************************************************
Starting task: Test Assemblies **\*test*.dll;-:**\xunit.runner.visualstudio.testadapter.dll
******************************************************************************
Executing the powershell script: C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\agents\default\tasks\VSTest\1.0.29\VSTest.ps1
Working folder: C:\a\1
Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "C:\a\1\s\artifacts\bin\project.test\Debug\dnx451\project.test.dll"  /EnableCodeCoverage /InIsolation /logger:trx
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
Warning: No test is available in C:\a\1\s\artifacts\bin\project.test\Debug\dnx451\project.test.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Attachments:
  C:\a\1\TestResults\0ad44b11-4dd4-41ba-a0bf-66d9cd487a3c\buildguest_TASKAGENT5-0007 2016-03-01 07_33_04.coverage
Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
No results found to publish.

The tests are written like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using Xunit;

namespace project.tests
{
    public class ProjectTests
    {
        [Fact]
        public void PassingTest()
        {
            Assert.Equal(4, Add(2, 2));
        }

        [Fact]
        public void FailingTest()
        {
            Assert.Equal(5, Add(2, 2));
        }

        int Add(int x, int y)
        {
            return x + y;
        }
    }
}

The tests run just fine locally, but not on Visual Studio Team Services.

For reference, this is my project.json

{
  "authors": [ "Tom Reiertsen" ],
  "commands": {
    "test": "xunit.runner.dnx"
  },
  "dependencies": {
    "project": "1.0.0-rc1-update1",
    "Microsoft.AspNet.Hosting": "1.0.0-rc1-final",
    "Microsoft.AspNet.TestHost": "1.0.0-rc1-final",
    "Microsoft.AspNet.WebApi.Client": "5.2.3",
    "Microsoft.CSharp": "4.0.1-beta-23516",
    "System.Collections": "4.0.11-beta-23516",
    "System.Linq": "4.0.1-beta-23516",
    "System.Runtime": "4.0.21-beta-23516",
    "System.Threading": "4.0.11-beta-23516",
    "xunit": "2.1.0",
    "xunit.runner.dnx": "2.1.0-rc1-build204",
    "xunit.runner.visualstudio": "2.1.0"
  },
  "description": "project.test Class Library",
  "frameworks": {
    "dnx451": { }
  },
  "licenseUrl": "",
  "projectUrl": "",
  "tags": [ "" ],
  "version": "1.0.0-rc1-update1"
}

Eddie Chen - MSFT · Answer 1 · 2016-03-02T03:25:01.027

1

Enter "$(Build.SourcesDirectory)\SolutionFolderName\packages" in "Advanced Execution Options\Path to Custom Test Adapters" to set the test adapter to xUnit. Refer to this link for details: xUnit or NUnit with Visual Studio Online Build

Update: I just create a sample app to test this and get the same issue with you. However, if I copy the xunit assembly files into "dnx451" folder together with test.dll, the test will be completed successfully.

And since you are working on a Asp.Net Core project and has imported DNX runner, you can also add a "Command Line" task to use dnx test command to run the testing directly.

edited Mar 02 '16 at 03:25

answered Mar 01 '16 at 09:54

Eddie Chen - MSFT

29,708
2
46
60

I tried "$(Build.SourcesDirectory)\test\packages" and "$(Build.SourcesDirectory)\test\project.test\packages" to no avail, the Visual Studio Test step would show an error. Then I noted where the packages were restored to and entered "C:\Users\buildguest\.dnx\packages\xunit.runner.visualstudio\2.1.0" and then the Visual Studio Test step succeeded but with the same warning as my initial problem (No tests found) so it still does not work. – TomRay74 Mar 01 '16 at 13:43
@TomRay74 I just create a ASP.Net core project for testing and can get the same issue with you. Please refer to my update answer for details. – Eddie Chen - MSFT Mar 02 '16 at 03:08
I am still not getting this to work. I tried copying the xUnit dll's into the the same folder as the test.dll but the problem remains the same. – TomRay74 Mar 04 '16 at 09:46
@TomRay74 The same as the files in my screenshot? – Eddie Chen - MSFT Mar 07 '16 at 08:52
And did you remove "$(Build.SourcesDirectory)\SolutionFolderName\packages" from "Advanced" after copying the xUnit dlls? – Eddie Chen - MSFT Mar 07 '16 at 10:00
Yes, I did copy all the files from your screenshot into the folder AND I removed the folder from the Advanced section but I still have the exact same message. I will try with a new blank project and see if the same happens there. – TomRay74 Mar 07 '16 at 10:37
Another two links for your reference: https://fluentbytes.com/running-xunit-test-with-asp-net-dnx-and-tfs-2015-build/ ; http://stackoverflow.com/questions/31870823/running-unit-tests-in-tfs-vso-build-vnext-using-xunit-adapter – Eddie Chen - MSFT Mar 08 '16 at 09:22
$(Build.SourcesDirectory)\packages\ will not work because the project.json NuGet mechanism will restore packages to your user temp directory, not \packages – oatsoda Sep 20 '16 at 09:41

score 0 · Answer 2 · answered Mar 04 '16 at 03:52

0

The tests are not getting discovered since the xunit test adapter for vstest was not found. Here is what you can do

add the xunit test adapter nuget reference to your test project
add the restore nuget packages as the first step in your build definition

vstest should not be able to detect your xunit test adapter and then discover and execute the xunit tests

answered Mar 04 '16 at 03:52

allen

4,627
1
22
33

I already have the xUnit NuGet packages installed in the project as you can see from my project.json dependencies above. – TomRay74 Mar 04 '16 at 09:49
The problem is that $(Build.SourcesDirectory)\packages\ will not work because the project.json NuGet mechanism will restore packages to your user temp directory, not \packages – oatsoda Sep 20 '16 at 09:41
@TomRay74 did you try updating your agent. Your current task version - 29 is pretty old. If thats not an option explicitly provide the xunit adapter path in the advanced options -> Path to Custom Test Adapters – allen Sep 21 '16 at 09:24

ben yip · Answer 3 · 2017-02-06T04:09:33.177

0

Someone posted this, i try this and it works http://blogs.perficient.com/microsoft/2016/08/unit-test-with-net-core-and-vsts/

In Execution Options -> Test Assembly, put **\test**\project.json
In Advance Execution Options -> Other Console Option, put /UseVsixExtensions:true /logger:trx

and the final result in visual studio team service

edited Feb 06 '17 at 04:09

answered Dec 20 '16 at 05:11

ben yip

117
1
6

This answer has nothing to do with xUnit and does not work for xunit. – Kasper Holdum Jan 27 '17 at 14:07
xUnit is the official test runner for Visual studio 2015. I tested this myself and works with xUnit. I have attached the screenshot to prove this is working – ben yip Feb 06 '17 at 04:07

No tests discovered using xUnit on ASP.Net Core 1.0 project with build on Visual Studio Team Services Hosted agent

3 Answers3