Run xUnit tests only on certain conditions

Question

I'm using xUnit and I have same load tests which I'd like to execute only under certain conditions:

Execute them always on our build server.
Do not execute them locally within the IDE when clicking Run all tests in the test runner.
Execute all load tests locally within the IDE if explicitly triggered.

How can I achieve this? Is there any chance to accomplish this with xUnit, preferably without conditional compiles?

`#if RELEASE` or `[System.Diagnostics.Conditional("RELEASE")` ? — TheGeneral, Sep 22 '21 at 09:17
@TheGeneral Thank you! I've modified my question since I'd like to avoid conditional compiles — mu88, Sep 22 '21 at 09:23
An answer from your side showing an applicable solution would be highly appreciated — mu88, Sep 22 '21 at 09:30
That would mean effort... However, someone else might be able to answer this without compiler directives or conditionals — TheGeneral, Sep 22 '21 at 09:31
related: https://stackoverflow.com/questions/4421328/how-do-i-skip-specific-tests-in-xunit-based-on-current-platform — Ruben Bartelink, Sep 22 '21 at 12:19
more techniques in https://stackoverflow.com/questions/14840172/skipping-a-whole-test-class-in-xunit-net but really there is no `ExplicitAttribute` in xUnit that would provide you a way to opt in even if the programmatic logic says no — Ruben Bartelink, Sep 22 '21 at 12:23

Asbjørn Thielke Gyring · Answer 1 · 2021-09-22T10:31:48.233

I think the easiest way to get around this if not the prettiest, is enclosing the test you want to "skip" locally is to enclose your test like this:


    public class TestClass1
    {
#if !DEBUG
         [Fact]
#endif
        public void Test1()
        {
            Assert.True(true);
        }
    }

This will skip the [Fact] attribute which triggers the test at run time, as long as you IDE or editor, or however you run your code, is set to build for "Debug", which is default for most.

A better way to handle this is probably using the xUnit [Fact(Skip = "Reason")] but I have had issues with making this work on runners on Gitlab sometimes.

And for your last case where you want it to run, you can just "run your code or tests" in "Release mode".

EDIT: In regards to your answer and my own suggestion of using the build in "skip" functionality for xUnit perhaps this post could be of help.

You could then create your own Attribute method to check the environment you run on:

https://josephwoodward.co.uk/2019/01/skipping-xunit-tests-based-on-runtime-conditions

Thank you for your solution. That's exactly the way how we solved it at the moment. But I really dislike the conditional compiles and because of that I'm trying to find another solution. But thank you anyways — mu88, Sep 22 '21 at 09:50
Yeah no worries. I completely agree this is a rather "brutish" solution. A better one would be to use skip reasons or perhaps use config files or some other way of running things on local "development" environments . — Asbjørn Thielke Gyring, Sep 22 '21 at 10:28

Run xUnit tests only on certain conditions

1 Answers1