Introduction:
I created a simple add function to get more insight in how Azure Dev Ops is working. When I let test_method2() fail on purpose the pipeline exists with ##[error]Bash exited with code '1'. Therefore in Azure Dev Ops an overview of Tests (which ones are fine and which ones are not) and Code Coverage is missing. I would expect that overview Test and Code Coverage is still provided in case of a failing test. To summarize, it's fine that a test fails but this should be reported and the process should not be stopped.
Note that when I repair the test, i.e. 'def test_method2(): assert add(15,5) == 20 ' the pipeline works as expected.
My question is what is needed to make sure that overview of Tests and Code Coverage is still provided in case of failing tests?
calc_lean.py:
def add(x, y):
"""Add Function"""
return x + y
test_calc_lean.py:
import pytest
from calc_lean import add
def test_method1():
assert add(3,5) == 8
def test_method2():
assert add(15,5) == 21
azure-pipelines.yml:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
strategy:
matrix:
Python37:
python.version: '3.7'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '$(python.version)'
displayName: 'Use Python $(python.version)'
- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
displayName: 'Install dependencies'
- script: |
pip install pytest
pip install pytest-cov
pytest --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml
displayName: pytest
- task: PublishTestResults@2
displayName: 'Publish Test Results **/test-results.xml'
inputs:
testResultsFiles: '**/test-results.xml'
testRunTitle: 'Python $(python.version)'
- task: PublishCodeCoverageResults@1
displayName: 'Code Coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'