3

We're using TFS 2010 Team Build which automatically runs our unit tests during the build process. If I click on View Log, I can see the list of tests run and each test result (Success/Fail). In the event of Failure, I want to see the test result output (so the developer can figure out what the cause of the failure is). Just running the unit test locally isn't enough because the issue could be environmental (if the test is failing because of some data, server, or physical path).

How can I change my build process template to include the full results of the unit test in the log?

MSTest command line from View Log page:

e:\Program Files\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"E:\Builds\1\1\Sources\Source\TestSettings.testsettings" /searchpathroot:"E:\Builds\1\1\Binaries" /resultsfileroot:"E:\Builds\1\1\TestResults" /testcontainer:"E:\Builds\1\1\Sources\Source\Testing\bin\Release\Testing.dll" /publish:"http://tfs:8080/tfs/Projects" /publishbuild:"vstfs:///Build/Build/196" /teamproject:"Project" /platform:"Any CPU" /flavor:"Release"

Screenshot of summary

http://imageshack.us/photo/my-images/28/tfsbuild.gif/

And of the Build Definition configuration

http://imageshack.us/photo/my-images/835/builddefinition.gif/

Thanks.

Jeff
  • 35,755
  • 15
  • 108
  • 220
  • 1
    Please refine what you actually have & what you would like to achieve. What unit-testing framework are you using? Do you get the 'View Test Results' in the Summary View? – pantelif Sep 21 '11 at 14:44
  • I'm using the built in mstest framework and activities that ship with TFS 2010 and are built in to the DefaultBuildProcessTemplate. I want the log of the build to include the full test result deatils for failed tests. – Jeff Sep 21 '11 at 15:39

2 Answers2

5

If you use the default template and run the test using the unit test framework provided by Microsoft, you should be able to see a link to the published test results on the View Summary page of the build details. When you click at this link, the test results will be downloaded from the database and put together in an trx file that will be displayed in your Test Window in Visual Studio.

UPDATE: The problem was figured out after looking at the logs. The build template was customized to use another tool to build instead of MSBuild, and the summary nodes for each configuration (platform/flavor) were not created as a result. That's why the build details view was missing the summary, including the test results.

Duat Le
  • 3,586
  • 2
  • 19
  • 18
  • I am using the default template. I do not see the published test results on the View Summary page. I only see the individual test results on the View Log page. – Jeff Sep 21 '11 at 15:40
  • We have modified the default template slightly...so I'm wondering if I screwed it up somehow...? I just checked the Run Tests activity and it does have publish set to true. And the tests ARE getting run and they're showing up in the View Log page... – Jeff Sep 21 '11 at 15:46
  • In the "View Log" page, do you see that the tests were published? At the end of the list of test results there should be a few lines that indicate the test results are publishing, and the last line should be "Published successfully". – Duat Le Sep 21 '11 at 17:00
  • 3
    If the link doesn't show up on the View Summary page, it's likely that the test results are published with a mismatched build configuration. For example, if your solution is compiled with Debug | Any CPU, but somehow the test results are published with Release | AnyCPU, then the link will not show up. Make sure the configuration matches case-sensitively and with no white spaces difference. – Duat Le Sep 21 '11 at 17:02
  • In View Log, I DO see Publish completed successfully...but nothing on the summary page...any thoughts? – Jeff Sep 21 '11 at 17:05
  • YES! Brilliant. I'm compiling in Release | Mixed Platform. The tests are Release | Any CPU. – Jeff Sep 21 '11 at 17:34
  • Ok, so that didn't fix it...but I think you've unearthed the problem here. We are not using msbuild to build the sln. We have vdproj files, so we're calling InvokeProcess on DevEnv... I've double checked that both DevEnv and the tests are being published as Release Any CPU...but I still don't see any results published in summary view. Any ideas? – Jeff Sep 21 '11 at 18:33
  • How did you double check? Did you look at the mstest.exe command-line arguments and verify that the build platform and flavor are "Any CPU" and "Release"? Can you take a screenshot of the build summary page, and share the mstest.exe arguments? – Duat Le Sep 21 '11 at 21:48
  • Screenshot to follow shortly...but the Log says /platform:"Any CPU" /flavor:"Release" in the mstest command line and the Build Definition says "Release" and "Any CPU" – Jeff Sep 21 '11 at 22:27
  • Your build summary is missing a bigger thing than just the test result link - it's the section under "Summary" that tells you the build results of your Platform | Flavor. I'm not sure why, do you mind sharing your TFS build log (diagnostics verbosity would be best) by copying the "View Log" page to a text file and upload it somewhere? – Duat Le Sep 22 '11 at 21:25
  • I noticed that too now... Sure, will do. Do you want the whole xaml file too? I'd rather not link to it from here. Do you have an email I can send it to? – Jeff Sep 22 '11 at 21:56
  • Sure. You can try "Email blog author" from this blog: http://blogs.msdn.com/b/duat_le/ – Duat Le Sep 23 '11 at 03:08
  • I sent them on your blog. Please feel free to post here or email directly. Thank you so much for your assistance. – Jeff Sep 23 '11 at 04:12
  • Hi Duat - what did you do to solve this? I also use NANT and my build summary page doesnt show much details. How did you make it display the results? – Isaiah4110 Sep 20 '13 at 15:42
4

There is one tiny, little detail that you need to know to get this to work. If you are using the web portal to view the test results for a build, You want to make sure that the Outcome column displays All (not just Failed). If all passed, nothing would be listed if the Outcome was set to Failed.

Here is a little picture of what I'm talking about: enter image description here

Notice how I have set the Outcome column to All. Now, if I select All as the Outcome, I can see all the tests listed. If you double-click any test the TFS web app will navigate you to the Run summary. Any tests that are linked with automation will have a Link provided to the Summary and any TRX files will be attached. Opening the TRX file will show the TestContext.WriteLine output, plus other data, like Duration.

Su Llewellyn
  • 2,660
  • 2
  • 19
  • 33