1

We have a complete Lab Management environment running Coded UI tests in nightly builds. What we are trying to achieve is to run our integration tests (regular TestMethod() with SQL connections) just before all the Coded UI tests to verify that our db scripts are executed correctly and that there are no new changes causing any problems.

So far I have found a way to execute tests remotely through .testrunconfig. The problem we have with that approach is that it's not possible to choose a testcontroller connected to a team project so I guess that would be only useful for running tests on physical machines outside of Lab Management?

One option seem to be to create a Test Case for each integration test and that should run it together with the UI tests but it feels like it will be to much maintenance managing hundreds of test cases just to run the integration tests. Also it would be better to completely separate the test runs for the different kinds of tests.

Is there any easy way to achieve this that I have totally missed? Or do I have to modify the lab build template to deploy and run the tests?

J. Horn
  • 215
  • 2
  • 5

2 Answers2

1

I guess that would be only useful for running tests on physical machines outside of Lab Management?

If you run your tests remotely through .testrunconfig you have to connect the Test Agent to another Test Controller which is NOT connected to the team project. Unfortunately it is impossible for the environment which are running under the Lab Management, to my knowledge.

What about this approach:

  1. Create an Ordered Test containing all you integration tests.
  2. Create a new Test Case "Integration Tests" and automate it by the ordered test So you do not have to maintain hundreds of Test Cases. You could also create several Ordered Tests if you want to group the integration tests and then create a "main" Ordered Test containing them. This way it will be easier to analyze test results especially if you have a lot of tests.
  3. Let the integration tests run as a part of your existing nightly build.
  4. Create a new Build Definition which does not start a build but uses the last successful nightly build and let your CodedUI tests run using Lab Build Template.

This way you will have different test runs for the different kinds of tests.

The only drawback is that you have to "synchronize" these two builds... You could just schedule the second build later so you could be sure the fist build is done.

It's not really perfect, I know... but this way you could easily achieve your goal.

Elena
  • 1,928
  • 1
  • 15
  • 23
0

I am not sure if there is an alternative solution, but on the project I am currently working on we have both our Unit and Integration Test Assemblies set under the Process options (Process>Basic>AutomatedTest>TestAssembly) in our Nightly Build.
This was achieved through altering the Default Build Process Template (not the Lab Default) a bit, as you suggested (I thought this was standard, but it's been a while).

Nashibukasan
  • 2,028
  • 23
  • 37
  • The problem here is that the tests will run on the build called from the lab build so they will execute before the deployment of the lab management servers and before our db scripts are executed. – J. Horn Apr 18 '13 at 10:48
  • Is that relevant? Unit and Integration tests are generally run without a Client or UI, so I do not see why they would need deployment. I feel I am missing something? Is the question that the tests are to run only just prior to Coded UI Tests? – Nashibukasan Apr 18 '13 at 11:09
  • During deployment we run the scripts that modify the database on the lab server and we want to run the integration tests to verify that the changes to the db hasn't caused any issues. By running the scripts on a db from a known snapshot we occasionally find some issues not found on the development db so it's quite handy from time to time. – J. Horn Apr 18 '13 at 14:10
  • I see! I will have a think about this then. It sounds like you will have to alter the Build Process Template. – Nashibukasan Apr 18 '13 at 22:13