0

I am looking for a tool either command line or GUI which copies the changed assemblies from the solution to a separated folder - so that a new build afterwards does not influence the test run. Afterwards it should executed a configurable set of tests (only certain assemblies, with filtering certain TestCategories). When it is finished the test results should be shown.

Is there a tool or a set of tools which does these tasks? MsTest.exe could run the test but not copy all necessary assemblies.

Using a combination of the MsTest command line tool (or the visual studio runner) in combination with post-build step to copy the assemblies locally are not good, because it would slow down every build, but I will not run the tests locally every time I build the solution. I could write a little script which copies the necessary assemblies locally beforehand. But what I was hoping for is a tool which does all this without me having to write a script.

Yggdrasil
  • 1,377
  • 2
  • 13
  • 27
  • possible duplicate of [How do I use MSTest without Visual Studio?](http://stackoverflow.com/questions/261290/how-do-i-use-mstest-without-visual-studio) – Panagiotis Kanavos Apr 23 '15 at 08:42
  • Not completely. See updated post above – Yggdrasil Apr 23 '15 at 09:16
  • 1
    The answers on the duplicate are out of date and are true for [Tag:VS2010]. Since the release of the new test runner in [TAG:vs2012] the new console runner is the way to go. – jessehouwing Apr 23 '15 at 09:18
  • Just add a post-build event to copy the files. You don't need an external tool for that. If you have a more complex scenario, look for a CI server like TeamCity to build your project and execute all tests in isolation whenever changes are commited – Panagiotis Kanavos Apr 23 '15 at 09:19
  • @jessehouwing the idea is the same - MSTest already has a console runner and files can be copied after each build. A CI/build server is a much better solution though and really simpler in the long run. – Panagiotis Kanavos Apr 23 '15 at 09:20
  • We have a build server. But I want to run my tests locally without impacting VS by running it inside VS. – Yggdrasil Apr 23 '15 at 09:24
  • @PanagiotisKanavos I agree that CI is a great use for this, but the question has merit. Being able to run tests on an environment from a script can be useful in many ways. – jessehouwing Apr 23 '15 at 10:26

2 Answers2

1

Copy files

To copy assemblies from the commandline you can use the standard copy command. To just copy "Changed" assemblies is harder, unless you're doing incremental builds.

xcopy, copy will suffice here.

MsBuild is the other tool that you can use to copy the files. You can create a post-build event or a custom target that doesn't run inside Visual Studio. It would only run when you set a specific condition or call the target explicitly from the commandline.

The handy fact is that MsBuild at least knows which files are required to build the files. Do note though, that MsBuild may now know exactly what is needed to run your tests. Certain config files and dependencies of 3rd party references may be needed too, but not part of the project.

I'm not aware of any other tools. I'd personally opt to write a simple script, as it isn't hard to do and maintain.

Run the tests

There is the commandline vstest.console.exe which will happily run the MsTest based tests.

The syntax looks like this:

 vstest.console.exe /TestCaseFilter:[ expression ] assemblyone.dll assembly2.dll

Note 1

Every time you run MsBuild to build your assemblies to build your project, even if all the MSIL generated is the same, your assembly will be different, as the compiler will assign a new unique GUID to the file.

So unless your build is incremental and detects that there is no need to actually build the file to begin with, it's going to generate unique files every time.

Note 2

Indeed, as other mention, Continuous Integration tools like or can help you build and run your tests and create a nice report for you.

More advanced tools such as or can run your tests during a deployment workflow when you're doing or even better .

jessehouwing
  • 106,458
  • 22
  • 256
  • 341
  • Thank you for your answer. This is kind of what I was looking for. I could differentiate the assemblies which were changed, with comparing the timestamp of the files. I thought of xcopy. But from which source. Each project has its own "bin/Debug" folder where all depended assemblies are located. So where do I get all necessary assemblies? – Yggdrasil Apr 23 '15 at 09:14
  • The standard trick is to add them as a reference to your test project and set them to copy-local. That way the `bin\$(configuration)` folder of your test project always has the correct set of files. – jessehouwing Apr 23 '15 at 09:17
  • This would make sense yes. Than I would only need to copy the content of all configured test assemblies, not coping files where I have already the most up-to-date version. – Yggdrasil Apr 23 '15 at 09:20
0

I think you are describing a team integration system.

You can use MS Team Build or the online version of it from visualstudio.com.

The other popular option is TeamCity

Both of these allow you to configure them so that they trigger a build that run your tests (or a subset of tests based on categories). If the build fails or if a test fails you have the option to react to this (e.g. reject the checkin)

Bogdan Gavril MSFT
  • 20,615
  • 10
  • 53
  • 74
  • I am looking for a solution which I can run locally. – Yggdrasil Apr 23 '15 at 09:17
  • You can run both TeamCity and TeamBuild locally if you have a powerful enough machine as these are more suited to a server enviroment. You gain three things this way: 1. You familiarize yourself with a continous integration system which most software shops use 2. If you grow your business you just buy a dedicated server to install TeamBuild / TeamCity and you're done 3. You don't mess about in msbuild which can be fiddly. – Bogdan Gavril MSFT Apr 23 '15 at 13:44