2

I am running our selenium based UnitTests on the WebUI thrue a different mechanism than the resular build steps in TFS 2015 (Update 1). Doing so generates a .trx file that I am trying to upload back into TFS so that the results are available in the build overview.

The build step running the UnitTests can fail the build if any of the tests fail, but then I would not have any test results visible in TFS. I can skip the failure by checking 'continue on error' in the step.

Next I can include a step 'Publish Test Results' that will pickup the generated trx file and results will be visible in the build.

But know the build doesn't fail!

Any idea's how to fail the build and have the testresults available?

I've checked:

  • vstest.console.exe (cannot publish testresults from a file)
  • MSTest (Enterprise/Premium license needed, we have MSDN Professional).

There has to be a better way. In my opinion, the build step 'Publish Test Results' should be able to check the file and fail the build?

Community
  • 1
  • 1
Rob Bos
  • 1,320
  • 1
  • 10
  • 25

1 Answers1

2

I use "Command Line" task to call "ParallelTestRunner.exe" to run the test during the build.

With this, if the test is failed without "Continue on error" option checked, the build will stop at this step, "Publish Test Results" step will not run which looks the same as the behavior you get. And if the test is failed with "Continue on error" option checked, the test step is marked as "Passed" and "Publish Test Results" step will run to publish the test result. But my final build status is not "Success", it is "Build Partially succeeded".

However, if you want to fail the test step and still run the "Publish Test Results" step, you can uncheck the "Continue on error" option for test step and check "Always run" option for "Publish Test Results" step. This will make sure the "Publish Test Results" step always run even though previous steps are failed.

By the way, "Publish Test Results" step only publish the test result, it does not check the result. The build is passed/failed base on the execution status of all the build steps in the build definition.

Eddie Chen - MSFT
  • 29,708
  • 2
  • 46
  • 60