15

I want to measure the Code Coverage of my XUnit-Tests in an ASP.NET Core application. The Tooling for .NET Core in Visual Studio 2015 is preview 2 and code coverage does not work so far.

The blog post http://dotnetthoughts.net/measuring-code-coverage-of-aspnet-core-applications-using-opencover/ from February shows a workaround by using the command line of open cover. I am looking for a more integrated way inside of Visual Studio.

Has anybody heard of a better / more integrated way of measuring Code Coverage in combination with XUnit ?

Ralf Bönning
  • 14,515
  • 5
  • 49
  • 67
  • fyi: OpenCover is used by the dotnet/corefx to get the coverage of their tests. – Shaun Wilde Jul 18 '16 at 20:24
  • @DavidOsborne - No, commercial tools are acceptable as well. I am eagerly waiting for the next Resharper EAP (early access preview). – Ralf Bönning Sep 20 '16 at 07:32
  • Then how about dotCover? I've also been impressed with NCrunch recently, which also reports coverage. – David Osborne Sep 20 '16 at 08:01
  • @DavidOsborne - Both do not have support for .net Core yet. For Resharper I hope it will be in the next version. NCrunch discusses this feature here https://ncrunch.uservoice.com/forums/245203-feature-requests/suggestions/8065623-support-dnx-projects?page=2&per_page=20 – Ralf Bönning Sep 20 '16 at 08:24
  • Ah. Didn't realise that. Sorry. – David Osborne Sep 20 '16 at 08:47
  • I blogged about how to write a self-contained script which calculates the test coverage of a .NET Core project with OpenCover: https://blog.markvincze.com/setting-up-coveralls-for-a-net-core-project/ – Mark Vincze Nov 17 '16 at 15:02

7 Answers7

7

Add the NuGet package Microsoft.CodeCoverage 1.0.1 to your project.json.

I am working on Template for Asp.NET and right now I am working on Unit Tests so I saw your post. You could see project/configuration here.

KernelPanic
  • 2,328
  • 7
  • 47
  • 90
adenial
  • 71
  • 1
  • 4
5

Disclaimer: These steps were given from Measuring ASP.NET Core Coverage with OpenCover - DotNetThoughts.

Even though the poster says about this site, I thought it would still be best to have these steps here for prosperity.

NOTE: These instructions while tailored to a windows operating system, should easily work for any O/S supported by OpenCover and ReportGenerator.

  1. Create your ASP.NET Core MVC Website
  2. Ensure global.json has "test" in projects
  3. Right click test folder in solution and add a new project
  4. Make sure the project type is an .NET Core Class library
  5. Add the following to your project.json dependencies node:
    • "dotnet-test-xunit": "2.2.0-preview2-build1029",
    • "xunit": "2.2.0-beta3-build3402"
    • "Microsoft.CodeCoverage": "1.0.2"
  6. Add the following to your project.json under version
    • "testRunner": "xunit",
  7. Write your unit tests
  8. Download OpenCover and Report Generator
  9. Install OpenCover
  10. Extract Report Generator into OpenCover install dir in folder called Report Generator
  11. Create a BAT file in your project and call it cover.bat
  12. Add the following contents:
@echo off

SET dotnet="C:\Program Files\dotnet\dotnet.exe"  
SET opencover="C:\Program Files (x86)\OpenCover\OpenCover.Console.exe"
SET reportgenerator="C:\Program Files (x86)\OpenCover\ReportGenerator\ReportGenerator.exe"

SET targetargs="test"  
SET filter="+[*]NAMESPACE.* -[*.Test]* -[xunit.*]* -[FluentValidation]*"  
SET coveragefile=Coverage.xml  
SET coveragedir=Coverage

REM Run code coverage analysis  
%opencover% -oldStyle -register:user -target:%dotnet% -output:%coveragefile% -targetargs:%targetargs% -filter:%filter% -skipautoprops -hideskipped:All

REM Generate the report  
%reportgenerator% -targetdir:%coveragedir% -reporttypes:Html;Badges -reports:%coveragefile% -verbosity:Error

REM Open the report  
start "report" "%coveragedir%\index.htm"
  1. Replace the NAMESPACE with your projects namespace.
  2. If more than one project, duplicate the regex +[*]NAMESPACE.* as many times as needed for each namespace
  3. Save the file
  4. Open a command prompt and ensure in your test project
  5. Type cover to get your unit tests running and your coverage results in HTML format, or whatever you named your bat file in step 11.
gunr2171
  • 16,104
  • 25
  • 61
  • 88
Wayne
  • 3,415
  • 1
  • 22
  • 21
5

The codecoverage works for me, for .Net Core using Microsoft.CodeCoverage as mentioned above.

Check you have Microsoft.CodeCoverage nuget added to your test project

Also check the project.json file on you main project, the debugType attribute should be "full" instead of "portable"

  "buildOptions": {
    "emitEntryPoint": true,
    "preserveCompilationContext": true,
    "debugType": "full"
  },

This fixed for me

Carlos Soria
  • 51
  • 1
  • 1
  • Awesome, thank you!! I was fighting this for so long. fwiw this is now `Full...` see also https://stackoverflow.com/questions/46902377/what-is-the-difference-between-using-debugtypefull-debugtype-and-debugtype – Lee Richardson Mar 21 '18 at 02:48
3

Coverlet (https://github.com/tonerdo/coverlet) is a new project that has recently emerged. It works alongside msbuild and gives a straight forward solution for coverage

Srivathsa Harish Venkataramana
  • 1,705
  • 21
  • 19
1

I got things working with some URL hints. (urls at the bottom)

This is "poor man's", but it will fire a htm file in your default browser with some good coverage htm reports. And much cheaper than VS 2017 ENTERPRISE version!

Obviously, you'll have to set your __unitTestProject to the correct value.

REM I call this file zzzCoverageRun.bat , or whatever name you want, but is a .bat file
REM PREREQUISITES
REM In Visual Studio, go to Tools / Nuget Package Manager / Package Manager Console
REM and run below line
REM dotnet tool install --global dotnet-reportgenerator-globaltool --version 4.0.15 


set __currentDirectory=%cd%

set __sln=%__currentDirectory%\..\Solutions\My.Solution.sln
set __unitTestProject=%__currentDirectory%\..\UnitTests\My.UnitTests.csproj


For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set __mydate=%%c-%%a-%%b)
For /f "tokens=1-3 delims=/:" %%a in ('time /t') do (set __mytime=%%a%%b%%c)
echo %__mydate%_%__mytime%


set __thisRunTag=%__localRepositoryName%%__mydate%_%__mytime%

set __coverageShortFileName=coverage.cobertura.xml
set __Common_TestResultsFileName=%TMP%\Zzz.TestResultsDirectory\%__thisRunTag%\%__coverageShortFileName%
REM set __ReportOutputDirectory=$(Build.SourcesDirectory)\TestResults\Coverage\Reports
set __ReportOutputDirectory=%TMP%\Zzz.CoverageResults\%__thisRunTag%
set __BuildConfiguration=Debug

REM build not needed
REM dotnet build "%__sln%"

dotnet test "%__unitTestProject%" --configuration %__BuildConfiguration% --logger:trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput="%__Common_TestResultsFileName%"

REM the below works because of the dotnet-reportgenerator-globaltool above
reportgenerator "-reports:%__Common_TestResultsFileName%" "-targetdir:%__ReportOutputDirectory%" -tag:%__thisRunTag% -reportTypes:htmlInline

start "" "%__ReportOutputDirectory%\index.htm"


set __coverageShortFileName=
set __currentDirectory=
set __ReportOutputDirectory=
set __sln=
set __unitTestProject=
set __Common_TestResultsFileName=
set __BuildConfiguration=
set __thisRunTag=

My UnitTests.csproj has this in it:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>

    <IsPackable>false</IsPackable>

    <AssemblyName>My.UnitTests</AssemblyName>

    <RootNamespace>My.UnitTests</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <None Remove="StyleCop.Cache" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="coverlet.msbuild" Version="2.6.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="1.3.2" />
    <PackageReference Include="MSTest.TestFramework" Version="1.3.2" />
    <PackageReference Include="Moq" Version="4.10.1" />
  </ItemGroup>

  <ItemGroup>
    <DotNetCliToolReference Include="dotnet-reportgenerator-cli" Version="4.0.15" />
  </ItemGroup>  



</Project>

Notable, the "coverlet.msbuild" and "dotnet-reportgenerator-cli". I know you need "coverlet.msbuild", the other one may just be an experimentation thing that you may or ma y not need.

I'm using alot of bat variables to drive things, and output to %TMP% folder to avoid VS/git wanting to check files in.

Hints came from:

https://holsson.wordpress.com/2018/11/30/test-code-coverage-with-net-core-and-tfs/

https://tattoocoder.com/cross-platform-code-coverage-arrives-for-net-core/

https://medium.com/bluekiri/code-coverage-in-vsts-with-xunit-coverlet-and-reportgenerator-be2a64cd9c2f

granadaCoder
  • 26,328
  • 10
  • 113
  • 146
0

I have just tried the ReSharper version 2016.3 EAP 4 (Early access preview). Besides unit test execution now dotCover returns code coverage information for my .net core test projects.

Ralf Bönning
  • 14,515
  • 5
  • 49
  • 67
  • 1
    Can you elaborate how you got that working? It does not work for me with 2016.3 EAP 4 (tests in separate .NET Core library) – Wayne Oct 20 '16 at 03:57
  • @Wayne - I did not do anything special. After installing all pieces of Resharper Ultimate I could execute the context menu "Cover" and see the results. So there was no special magic in my case. – Ralf Bönning Oct 20 '16 at 12:24
  • Interesting, that just does not work for me, my dotcover just hangs on pending. Could you update your answer with a description of your solution setup and project.json? I will create an answer for OpenCover as an option for those without a jetbrains subscription – Wayne Oct 21 '16 at 00:09
  • This now works in EAP6 - https://youtrack.jetbrains.com/issue/RSRP-461198 – Wayne Oct 30 '16 at 21:47
0

OpenCover+ReportGenerator. Install as NUGET packages to one of your test projects (once, because they just need to appear in package folder).

Then you can put this powershel https://github.com/rpokrovskij/opencover4vs.ps1 to your solution folder, and setup it with your Test projects using path globing e.g.

$TestProjectsGlobbing = @(,'*.Test.csproj')

Then you can run it as a regular Powershel script. Result should appear in Solution Folder in \TestsResults\report\index.html

Roman Pokrovskij
  • 9,449
  • 21
  • 87
  • 142