How to run some tests only in azure devops CI env but not locally

Question

I would like to specify some long-running c# project xUnit tests to only run in azure devops CI pipeline, but not when a click 'Run All' in Visual Studio locally (vs2019). Is there a 'best practice' for this kind of behaviour?

I played around with making a test playlist and running that locally instead of Run All, but that would require updating the list (or lists) everytime i add new tests and this is error-prone.

Leo: I ended up with Ruben's solution, making a custom `FactAttribute` class that sets `Skip` property if `TF_BUILD` is not set in environment. DRY, does not hide tests from explorer and expresses intent cleanly. — anonymousdotnetdeveloper666, Jun 17 '19 at 07:34
If you want to solve this for **NUnit** rather than **xUnit**, see over here: https://stackoverflow.com/questions/62388204/how-to-run-some-nunit-tests-only-in-azure-devops-ci-env-but-not-locally?noredirect=1&lq=1 — Brondahl, Jun 15 '20 at 12:26

score 7 · Answer 1 · edited Jun 20 '20 at 09:12

7

AzureDevOps defines some environment variables. I would use TF_BUILD in conjunction with Ruben Bartelinks answer.

Set to True if the script is being run by a build task.

This variable is agent-scoped. It can be used as an environment variable in a script and as a parameter in a build task, but not as part of the build number or as a version control tag.

edited Jun 20 '20 at 09:12

Community

1
1

answered Jun 14 '19 at 17:59

Sebastian Segerer

370
1
6

This is working well. Getting the value with a `Environment.GetEnvironmentVariable("TF_BUILD")` and comparing it to "True" does separate execution environment from Azure DevOps pipeline and others. – Jari Turkia Oct 28 '19 at 13:57

score 6 · Accepted Answer · answered Jun 14 '19 at 17:03

The Skip property in FactAttribute is overridable - you can make a FactWhenAttribute and have it check an environment variable

Example in this post from @Joseph Woodward

public sealed class IgnoreOnAppVeyorLinuxFact : FactAttribute
{
    public IgnoreOnAppVeyorLinuxFact() {
        if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux) && IsAppVeyor()) {
            Skip = "Ignore on Linux when run via AppVeyor";
        }
    }

    private static bool IsAppVeyor()
        => Environment.GetEnvironmentVariable("APPVEYOR") != null;
}

Ah, this combined with Sebastian's answer would be a clean solution! — anonymousdotnetdeveloper666, Jun 17 '19 at 07:22

score 1 · Answer 3 · answered Jun 14 '19 at 07:36

Assuming your build pipeline is building using the Release build configuration and not using the Debug build config, you could use the #IF !DEBUG preprocessor directive arround your [Fact] attribute.

For example:

#if !DEBUG
[Fact]
#endif
public void YourTestMethod()
{
}

How to run some tests only in azure devops CI env but not locally

3 Answers3

Linked