2

in our company, we have many UI tests which are arranged in playlists (*.orderedtest file). In release definition are running this tests like ordered tests: VSTest_configure.

After running tests, the test results are displayed only like one test result per one playlist (*.orderedtest): Test Results. But all of the tests from the playlists was running. When I opened the *.trx file for one *.orderedtest (e.g. ListFrame1) in Visual Studio, I can see test results: TestResultTRX and after doubleclick on the test name "playlist_listframe1" I can see all of the test results from playlist ListFrame1: Results from all tests of the playlist ListFrame1.

Do you know anybody why the results of all tests from playlist are not displayed in test results on TFS web? This issue began to emerge after TFS was updated on the new version of tfs 2018 update 3.2 and resave the release definition with VSTest tasks. In the VSTest task is now in the "Advanced execution options" the new settings "Batch tests". Is it possible, that this new functionality changed publishing or creating test results? The expected state is that the results should be displayed for all of the tests from the playlists. Just like before updating TFS 2018 update 3.2: Expected Test Results

Please, give me an advice. Thank you.

SuperMario
  • 21
  • 3
  • did you find any solution to the problem described here? I am having the same problem – Zankhana Rana Feb 25 '20 at 15:02
  • 1
    Hi, I have not found any solution to this issue. We had to rework the tests to make them independent of order or data. In addition, we added the "TestCategory" attribute to all tests, which allowed us to sort tests into different "Playlists". Now we do not run tests as * .orderedtest file, but we run one VSTest task for each "TestCategory". In each VSTest task we put into the "Test files" directly compiled dll assembly with tests and in "Test filter criteria" we put TestCategory of each tests. – SuperMario Feb 27 '20 at 06:34

0 Answers0