1

Introduction:

I created a simple add function to get more insight in how Azure Dev Ops is working. When I let test_method2() fail on purpose the pipeline exists with ##[error]Bash exited with code '1'. Therefore in Azure Dev Ops an overview of Tests (which ones are fine and which ones are not) and Code Coverage is missing. I would expect that overview Test and Code Coverage is still provided in case of a failing test. To summarize, it's fine that a test fails but this should be reported and the process should not be stopped.

Note that when I repair the test, i.e. 'def test_method2(): assert add(15,5) == 20 ' the pipeline works as expected.

My question is what is needed to make sure that overview of Tests and Code Coverage is still provided in case of failing tests?

calc_lean.py:

def add(x, y):
    """Add Function"""
    return x + y

test_calc_lean.py:

import pytest
from calc_lean import add

def test_method1():
    assert add(3,5) == 8

def test_method2():
    assert add(15,5) == 21 

azure-pipelines.yml:

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'
strategy:
  matrix:
    Python37:
      python.version: '3.7'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '$(python.version)'
  displayName: 'Use Python $(python.version)'

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  displayName: 'Install dependencies'

- script: |
    pip install pytest 
    pip install pytest-cov
    pytest --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml 
  displayName: pytest

- task: PublishTestResults@2
  displayName: 'Publish Test Results **/test-results.xml'
  inputs:
    testResultsFiles: '**/test-results.xml'
    testRunTitle: 'Python $(python.version)'

- task: PublishCodeCoverageResults@1
  displayName: 'Code Coverage'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
EvR020
  • 21
  • 1
  • 1
  • 9
  • 1
    If you found a solution can you add it as reply. This will be good to know that your problem is already solved. – Krzysztof Madej May 22 '20 at 09:00
  • Hi EvR2020, You could move your solution to below as a new reply and [mark it as an answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work), which will also help others in the community. – PatrickLu-MSFT May 25 '20 at 05:10

1 Answers1

1

Solution: Problem was already discussed in other posts. Amongst others Failed Cypress test exit in 'bash exited with code 1' in Azure DevOps

Adding ' || true' adressed it. That is,

pytest --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml  || true

supresses the err

EvR020
  • 21
  • 1
  • 1
  • 9