How to exclude certain tests in the Visual Studio Test Runner?

Question

I have attributes on certain tests that I ideally don't want to run on every build. Most of my tests are normal unit tests and I do want them to run on every build.

So: how can I exclude a test by category or project type?

For example, I'd like to exclude CodedUItests:

[CodedUITest]
public class SearchViewTests

...or exclude tests in a given TestCategory:

[TestMethod]
[TestCategory("Database Integration")]
public void ContactRepositoryGetByIdWithIdExpectCorrectContact()

I particularly want to exclude the coded UI tests as they disrupt my ability to continue working, whereas all the other tests will happily run in the background without disturbing me.

Originally this question was about Visual Studio 2012, so I'd prefer solutions that work in that version and higher.

score 76 · Answer 1 · edited Dec 02 '22 at 10:04

TL;DR version:

Test explorer showing -Trait:"CategoryName" filter

Other answers have commented on workarounds and use of the more recent Traits options. However, none quite tell you how to specifically exclude tests for a trait. To do so, simply use a - (minus) to negate a filter in the search box, e.g.:

-Trait:"DatabaseIntegration"

This will exclude all tests with that trait. The MSDN documentation on these features has the following explanation:

To exclude a subset of the results of a filter, use the following syntax:
FilterName:"Criteria" -FilterName:"SubsetCriteria"
For example,
FullName:"MyClass" - FullName:"PerfTest"
returns all tests that include "MyClass" in their name except those tests that also include "PerfTest" in their name.

I wish it had a wildcard feature -trait:"%Criteria" but alas wishful thinking — rjt011000, Apr 12 '19 at 18:42

score 8 · Answer 2 · answered Oct 10 '12 at 05:59

The Visual Studio 2012 Update 1 CTP has added support for grouping by projects and categories (traits) to the Test Explorer. http://blogs.msdn.com/b/visualstudioalm/archive/2012/10/08/visual-studio-2012-update-1-ctp.aspx http://go.microsoft.com/fwlink/?LinkId=268021

score 7 · Accepted Answer · answered Oct 03 '12 at 08:43

7

The only "solution" (or better workaround) I found to work is to specify a "FullName" filter. Basically I usually structure my solution like

ProjectA
ProjectA.UnitTests
ProjectA.IntegrationTests

and so on. Now I can specify a filter in the Test Explorer like FullName: "UnitTests" which seems to work.
(I'd expected to be able to use a regex within the search name but it doesn't seem to be supported.)

enter image description here

answered Oct 03 '12 at 08:43

Juri

32,424
20
102
136

Todd's note of the feature coming soon is great news - but this is the interim solution that works best for me as I can filter `FullName:"MyProject.Integration.Tests"` to just see the integration tests and `FullName:"MyProject.Tests"` to see the unit tests. – Fenton Oct 12 '12 at 14:03

score 3 · Answer 4 · edited Sep 26 '12 at 23:32

You can use the search filter in Unit Text Explorer.

unit test explorer

In the new Test Exlorer shipped with 2012, you can group tests by 'Test Results', 'FileName',... etc but in RTM build group/search by 'TestCategory' is not present from Test Explorer. However there is a filter in commandline and TeamBuild based on TestCategory (Running selective unit tests in VS 2012 RC using TestCaseFilter) in VS2012.

From a member of VS dev Team, they understood the importance of this missing feature. It is in the backlog, and it should be available in furutre builds/updates.

The following blog provides more details: http://blogs.msdn.com/b/vikramagrawal/archive/2012/07/23/running-selective-unit-tests-in-vs-2012-rc-using-testcasefilter.aspx

score 3 · Answer 5 · answered Dec 04 '12 at 18:10

3

There's a filter you can apply on the Test Explorer. Trait:"UnitTest" this filter will pickup only the tests with TestCategory("UnitTest") attribute ignoring the others.

answered Dec 04 '12 at 18:10

Waldyr Felix

39
3

score 2 · Answer 6 · answered Jul 15 '16 at 13:19

A bit late to the partyI had the same issue where all developers were confronted with disruptive CodedUI when just running all unit tests in a solution. My workaround is to add the following to the AssemblyInitializer of the CodedUI Project:

    if(!ConfigurationManager.AppSettings["MachinesToRunCodedUI"].Split(',').Contains(Environment.MachineName))
        Assert.Inconclusive("CodedUI Tests are skipped.");

Only when a machine is part of that MachinesToRunCodedUI list, it will run the CodedUI tests.

score 0 · Answer 7 · answered Sep 25 '12 at 14:19

0

Have you seen this article? How to: Group and Run Automated Tests Using Test Categories

I haven't yet tried this with VS 2012, but it still might be valid. As far as I could see there are no references to TFS, so this should be pointing at the normal VS test runner.

answered Sep 25 '12 at 14:19

Jens H

4,590
2
25
35

2

I don't think this is in VS2012 - the Test View has been replaced by Test Explorer and it doesn't seem to be category aware - although Cybermaxs answer suggests the VS Dev Team may be adding it. The VS2012 equivalent article suggests it is only available via command line currently: http://msdn.microsoft.com/en-us/library/dd286683%28v=vs.110%29.aspx – Fenton Sep 25 '12 at 15:58

How to exclude certain tests in the Visual Studio Test Runner?

7 Answers7

Linked