7

I need guidance in generating code coverage report of Asp.net unit tests in azure build pipeline. My project is based on .Net Framework 4.6.

I am able to run all the unit tests using "visual studio test" task.

I tried the "report generator" task, but it require cobertura or jacoco etc xml files, which am unable to generate in the build pipeline.

Expectation - I want to get code coverage report for the runned unit tests which will show complete information like the lines coverage, branch coverage, function coverage etc. same as what "report generator" generates.

Note: I am able to generate the reports using opencover and reportgenerator on my local system but am unable to find a way to do the same in azure build pipeline.

Shubho
  • 135
  • 1
  • 8
  • You can use coverlet as explain in this post: [Computing code coverage for a .NET Core project with Azure DevOps and Coverlet](https://www.meziantou.net/2019/02/18/computing-code-coverage-for-a-dotnet-core-project-with-azure-devops-and-coverlet) – meziantou Jun 09 '19 at 19:18
  • Thank you for the reply, @meziantou. My project is based on **.net framework 4.6** and not **.net core**. I have updated the same in the description. I apologize for not mentioning the same before. Just want to confirm can I use **coverlet** in **.net framework 4.6**. If yes, then which azure build task I need to use. – Shubho Jun 10 '19 at 18:44

2 Answers2

16

To get the Code Coverage results in .Net framework you just need to enable it in the "Visual Studio Test" task:

enter image description here

If you are use .yml builds the syntax is:

- task: VSTest@2
  inputs:
    codeCoverageEnabled: true

Results:

enter image description here

Note: if you use Microsoft Hosted Agent you will see the results, if you use Self Hosted Agent you must Visual Studio Enterprise version to see the Code Coverage results.

If you want more detailed code coverage report you can use coverlet in .Net framework by install the tool during the pipeline and then generate the report. you can do with a PowerShell script:

dotnet tool install dotnet-reportgenerator --tool-path . --version 4.0.12
dotnet tool install coverlet.console --tool-path . --version 1.4.1
mkdir .\reports
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*test*.dll" }
$coverlet = "$pwd\coverlet.exe"
& $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"
gci -Recurse |
?{ $_.Name -eq "coverage.cobertura.xml"} |
%{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" }

enter image description here

Then add "Publish code coverage" task with these parameters:

enter image description here

Results:

enter image description here

Shayki Abramczyk
  • 36,824
  • 16
  • 89
  • 114
  • 1
    Thank you very much @Shayki. Initially, when I tried to run your code in azure devops, I was getting the error "**Unable to find package dotnet-reportgenerator**". But when I changed **"dotnet-reportgenerator"** in _"dotnet tool install dotnet-reportgenerator --tool-path . --version 4.0.12"_ to **"dotnet-reportgenerator-globaltool"**. It started working. – Shubho Jun 10 '19 at 18:29
  • However @Shayki, Out of curiosity just want to know do we need `gci -Recurse | ?{ $_.Name -eq "coverage.cobertura.xml"} | %{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" }` in powershell script as **Publish code coverage results task** will publish the reports anyway. – Shubho Jun 10 '19 at 18:31
  • @Shubho You're welcome! thanks for your comment! I don't really know why, I just saw it in elsewhere and it worked for me :) – Shayki Abramczyk Jun 10 '19 at 18:52
  • No problem, @Shayki. I have one more requirement. I need to display some of the code coverage information in email too. Can you tell me whether there is any api through which I can get the generated coverage information, if not full atleast some part of it. I have tried (https://learn.microsoft.com/en-us/rest/api/azure/devops/test/code%20coverage/get%20build%20code%20coverage?view=azure-devops-rest-5.0) but not getting any information – Shubho Jun 10 '19 at 18:59
  • Please open a new question exactly what you did you try. – Shayki Abramczyk Jun 10 '19 at 19:04
  • Ok @Shayki. Will do that. In case any more help is required regarding this question will comment the same here. Thanks once again. – Shubho Jun 10 '19 at 19:06
  • Hi @Shayki, we accomplished this task using the .Net Core CLI. Is there a way to accomplish this task (generation of code coverage) without using the .Net Core CLI. I mean without installing .Net Core on the build server. – Shubho Jun 10 '19 at 19:26
  • Do you want the detailed coverage or the basic? – Shayki Abramczyk Jun 10 '19 at 19:28
  • Yes @Shayki, I need the detailed coverage. – Shubho Jun 10 '19 at 19:38
  • @Shubho You need to include corbertura or Jacoco in your project, google it. in the current situation you can download the basic results and open it in the VS then you will see the all details. – Shayki Abramczyk Jun 11 '19 at 09:46
  • Okay. @Shayki. Actually my development server doesn't have .Net Core installed, that's why I want to avoid installing the same, and also I don't want to make any change in the project for generation of code coverage. – Shubho Jun 11 '19 at 13:50
  • Why now stay with .net core? – Shayki Abramczyk Jun 11 '19 at 13:54
  • It is development server and I can't make changes as per my wish without approval. If at all it is not possible without .Net Core then will think of going with it. – Shubho Jun 11 '19 at 13:59
  • Currently, I am thinking of downloading OpenCover or any other tool on the build server and run the same exe from the build pipeline to generate the cobertura or jacoco file and then consume the output in Report Generator task of the build pipeline. I think this should work. _I am interested in knowing if there is any way by which I can download opencover.exe from build pipeline without downloading it externally_. – Shubho Jun 11 '19 at 17:27
  • I don’t know how :/ but what is the difference between .net core to open cover? Both you want to download during the build. – Shayki Abramczyk Jun 11 '19 at 19:20
  • Actually, I am unable to find a way to download **opercover** from build pipeline, so will have to download it once on the build server before triggering build. Whereas, in **.Net Core** we are able to download **coverlet** using **.Net Core CLI** during build. – Shubho Jun 12 '19 at 04:58
  • So why you prefer opencover rather them .net core + coverlet (that works with regular .net framework)? – Shayki Abramczyk Jun 12 '19 at 06:26
  • Only want to avoid any extra installation on the development server. That's it, nothing else. And also want to know, if there is any other way out or not. – Shubho Jun 12 '19 at 17:28
  • But you want to install opencover so it also extra installation.. no? – Shayki Abramczyk Jun 13 '19 at 05:57
  • Yes it is. But for generating code coverage I need to install something, either opencover or coverlet or any other. – Shubho Jun 13 '19 at 06:19
  • Hi @ShaykiAbramczyk and Shubho, I tried the PS script shared by Shayki Abramczyk but it is failing with error "The filename or extension is too long" error. There are multiple test files\dll for which i want to generate the coverage file. I am getting error like below-- Program 'coverlet.exe' failed to run: The filename or extension is too longAt E:\Cache\Agent03\18\s\codecoverage.ps1:8 char:1 + & $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vs – SRP Oct 01 '21 at 15:09
  • This is the script i am using --- CD $PSScriptRoot dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.12 dotnet tool install coverlet.console --tool-path . --version 1.4.1 mkdir .\reports1 $unitTestFile = gci -Recurse | ?{ $_.FullName -like "*test*.dll" } Write-Host "$unitTestFile" $coverlet = "$pwd\coverlet.exe" & $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura" – SRP Oct 01 '21 at 15:10
4

For anyone looking for code coverage in Azure Devops (using classic editor, without Yaml), in current .NET (core) 5, with xUnit tests:

  1. In your xUnit test project, add following (it generally comes by default in .NET 5, xUnit template now):

    \<PackageReference Include="coverlet.collector" Version="3.0.3" /\>

    Keep checking for new version.

  2. Head to Azure devops, create pipeline using classic editor. Do the restore, build steps. (Or you can choose dotnet core template as below): enter image description here

  3. In the test command of dotnet core task, add argument - --collect:"XPlat Code Coverage". Remember "XPlat Code Coverage" is friendly name and case sensitive. Your test command would look like: enter image description here Check this checkbox if you want to publish your test results: Publish test results and code coverage, but it won't publish code coverage. The functionality is not yet working (at least not in non-windows).

  4. Next add - Publish code coverage results task. Choose "Code coverage tool" as "Cobertura" and in "Summary file" field, add $(Agent.TempDirectory)/**/coverage.cobertura.xml. Looks like this: enter image description here

  5. Save and Queue (in any agent, I use Ubuntu) and see the result once pipeline run completes: enter image description here

Meer
  • 678
  • 6
  • 12