What is required for doing Unit Tests on a Build Server?

Question

I am doing unit tests (no additional frameworks other then what comes out of the box with VS 2013).

It all works locally. What do I need to do to get it working on the build server? The build server is a TFS 2013. There are no visual studio installed other then an 2010 version. I have already moved all the 4.5 .net stuff and .targets files from my visual studio to the build server to be able to compile 4.5 dlls and azure projects.

First I had problem with the build template was using AgileRunner or something thats coming with VS2013. I changed that to MSTest. Is this the correct change? Is there any installers for MSTest only without visual studio that I can install on the build server?

My boss prefers if I can avoid installing Vs2013 on the build server.

Answer 1

Just install Visual Studio on the build server; It is a very common practice. The requirement is mentioned in the installation guidelines. You can have visual Studio 2010 and 2013 installed side-by-side if that is desired. There are so many parts of .NET that depend on it.

Install Visual Studio and other software to enable compilation and other capabilities

You must install on the build agent the version of Visual Studio that your team uses on its dev machines. See Installing Visual Studio. You must also install any other software and components that are installed on your dev machines and that are required to build your app.

^{http://msdn.microsoft.com/en-us/library/bb399135.aspx#software}

You're even allowed to use an existing license for the build server, so there are no additional costs involved if you own an MSDN subscription:

Using Visual Studio on the Build Server

If you have one or more licensed users of Visual Studio Ultimate with MSDN, Visual Studio Premium with MSDN, or Visual Studio Professional with MSDN, then you may also install the Visual Studio software as part of Team Foundation Server 2013 Build Services. This way, you do not need to purchase a Visual Studio license to cover the running of Visual Studio on the build server for each person whose actions initiate a build.

^{http://www.microsoft.com/en-us/download/details.aspx?id=13350}

Is there a specific reason for you boss to not want Visual Studio to be installed?

With regards to the Agile Test Runner, it is the improved test runner that shipped with Visual Studio 2012, it's the replacement of the old MsTest runner, and will replace the old MsTest runner for the last few features for which the old runner is still needed. Though the MsTest runner will still work, certain features will not be as easy to use and confugure (like Code Coverage gathering). The Agile test runner is also required for other test framwork extensibility options, including XUnit .NET, NUnit, but also Javascript Unit Test support (using Chutzpah).

Other features that will require Visual Studio to be installed include:

• Code Analysis
• Code coverage
• MsDeploy / WebDeploy
• SQL Server Data Tools
• and others.

As an alternative to get just the test runners, you could attempt to install the TFS Test Controller and Agent to the server (you don't need to configure them), the Test Agent will install a number of testing related features without actually installing the Visual Studio Shell.

Answer 2

By not installing Visual Studio on your build server, you are violating the license agreement with Microsoft. Although Microsoft themselves might build without installing Visual Studio on their build servers internally, this scenario is definitely not supported officially.

Building without installing Visual Studio is asking for undefined behavior and jeopardizing the integrity of your builds. I would suggest to stay clear from these attempts until Microsoft officially supports this scenario.

Microsoft strongly recommended not to build without installing Visual Studio when we asked as Gold Partner 2 years back as it was not supported in any way officially and there is no license model covering the scenario.