21

I have some tests that I would like to ignore when using cargo test and only run when explicitly passed a feature flag. I know this can be done by using #[ignore] and cargo test -- --ignored, but I'd like to have multiple sets of ignored tests for other reasons.

I have tried this:

#[test]
#[cfg_attr(not(feature = "online_tests"), ignore)]
fn get_github_sample() {}

This is ignored when I run cargo test as desired, but I can't get it to run.

I have tried multiple ways of running Cargo but the tests continue to be ignored:

cargo test --features "online_tests"

cargo test --all-features

I then added the feature definition into my Cargo.toml as per this page, but they continue to be ignored.

I am using workspaces in Cargo. I tried adding the feature definition in both Cargo.toml files with no difference.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Andrew Mackenzie
  • 5,477
  • 5
  • 48
  • 70
  • 1
    Have you tried just using `#[cfg(feature = "online_tests")]` on the respective tests? – Neikos Feb 02 '18 at 14:48
  • 1
    @Neikos the downside to that is that the tests won't even be compiled. If there was some compilation error, you wouldn't see it until those features were enabled. Also, you won't see that there are ignored tests in the test output, potentially reminding you to enable them. – Shepmaster Feb 02 '18 at 15:36
  • I understand that they need #[test] also and if I did that I understand that "cargo test" would run them no? And that's not what I want. – Andrew Mackenzie Feb 02 '18 at 15:39

1 Answers1

26

Without a workspace

Cargo.toml

[package]
name = "feature-tests"
version = "0.1.0"
authors = ["An Devloper <an.devloper@example.com>"]

[features]
network = []
filesystem = []

[dependencies]

src/lib.rs

#[test]
#[cfg_attr(not(feature = "network"), ignore)]
fn network() {
    panic!("Touched the network");
}

#[test]
#[cfg_attr(not(feature = "filesystem"), ignore)]
fn filesystem() {
    panic!("Touched the filesystem");
}

Output

$ cargo test

running 2 tests
test filesystem ... ignored
test network ... ignored

$ cargo test --features network

running 2 tests
test filesystem ... ignored
test network ... FAILED

$ cargo test --features filesystem

running 2 tests
test network ... ignored
test filesystem ... FAILED

(some output removed to better show effects)

With a workspace

Layout

.
├── Cargo.toml
├── feature-tests
│   ├── Cargo.toml
│   ├── src
│   │   └── lib.rs
├── src
│   └── lib.rs

feature-tests contains the files from the first section above.

Cargo.toml

[package]
name = "workspace"
version = "0.1.0"
authors = ["An Devloper <an.devloper@example.com>"]

[features]
filesystem = ["feature-tests/filesystem"]
network = ["feature-tests/network"]

[workspace]

[dependencies]
feature-tests = { path = "feature-tests" }

Output

$ cargo test --all

running 2 tests
test filesystem ... ignored
test network ... ignored

$ cargo test --all --features=network

running 2 tests
test filesystem ... ignored
test network ... FAILED

(some output removed to better show effects)

With a virtual workspace

Virtual workspaces do not support specifying features (Cargo issue #4942). You will need to run the tests from within the sub project or specify the path to the appropriate Cargo.toml

Layout

.
├── Cargo.toml
└── feature-tests
    ├── Cargo.toml
    └── src
        └── lib.rs

feature-tests contains the files from the first section above.

Cargo.toml

[workspace]
members = ["feature-tests"]

Output

$ cargo test --all --manifest-path feature-tests/Cargo.toml --features=network 

running 2 tests
test filesystem ... ignored
test network ... FAILED

$ cargo test --all --manifest-path feature-tests/Cargo.toml

running 2 tests
test filesystem ... ignored
test network ... ignored

(some output removed to better show effects)

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366