Generating unit tests for pairs of files on disk

Question

A few questions (such as How can I create parameterized tests in Rust?) deal with using macros to create parameterised unit tests in Rust. I need to use this technique to generate a pair of unit tests for every pair of input files in a directory. The unit tests themselves just call a simple function:

fn check_files(path1: &str, path2: &str, msg: &str) {
    assert!(true, "FAILURE: {}: {} and {}.", msg, path1, path2);
}

I use lazy_static to generate a list of input files:

#![feature(plugin)]
#![plugin(interpolate_idents)]

extern crate glob;
#[macro_use]
extern crate lazy_static;

use glob::glob;

lazy_static! {
    /// Glob all example files in the `tests/` directory.
    static ref TEST_FILES: Vec<String> = glob("tests/*.java")
        .expect("Failed to read glob pattern")
        .into_iter()
        .map(|res| res.unwrap().to_str().unwrap().to_string())
        .collect::<Vec<String>>();
}

And then the macros use the interpolate idents crate to concatenate identifiers to create the unit test names:

#[test]
fn test_glob_runner() {
    // Define unit tests for a single pair of filenames.
    macro_rules! define_tests {
        ($name1:tt, $name2:tt, $fname1:expr, $fname2:expr) => ( interpolate_idents! {
            #[test]
            fn [test_globbed_ $name1 _ $name2 _null]() {
                check_files($fname1, $fname2, "null test");
            }
            #[test]
            fn [test_globbed_ $name1 _ $name2 _non_null]() {
                check_files($fname1, $fname2, "non-null test");
            }
        } )
    }
    // Write out unit tests for all pairs of given list of filenames.
    macro_rules! test_globbed_files {
        ($d:expr) => {
            for fname1 in $d.iter() {
                for fname2 in $d.iter() {
                    // Remove directory and extension from `fname1`, `fname2`.
                    let name1 = &fname1[6..].split(".").next().unwrap();
                    let name2 = &fname1[6..].split(".").next().unwrap();
                    || { define_tests!(name1, name2, fname1, fname2) };
                }
            }
        }
    }
    // Test all pairs of files in the `tests/` directory.
    test_globbed_files!(TEST_FILES);
}

This gives the following compiler error:

error: expected expression, found keyword `fn`
  --> tests/test.rs:14:13
   |
14 |             fn [test_globbed_ $name1 _ $name2 _null]() {
   |             ^^

This error message makes little sense to me, not least because the define_tests macro is similar to the code here. However, I'm not sure that it's really possible to use name1 and name2 in the unit test name.

There is a complete but simplified example project on GitHub, just clone and run cargo test to see the compiler error.

Answer 1

The trouble with your attempted approach at parameterized tests is that TEST_FILES is computed only at runtime, while you are expecting to be able to use it at compile time to stamp out the several #[test] functions.

In order to make this work, you will need some way to compute TEST_FILES at compile time. One possibility would be through a build script that iterates the glob at build time and writes out #[test] functions to a file that can be included from your test directory.

In Cargo.toml:

[package]
# ...
build = "build.rs"

[build-dependencies]
glob = "0.2"

In build.rs:

use std::env;
use std::fs::File;
use std::io::Write;
use std::path::Path;

extern crate glob;
use glob::glob;

fn main() {
    let test_files = glob("tests/*.java")
        .expect("Failed to read glob pattern")
        .into_iter();

    let outfile_path = Path::new(&env::var("OUT_DIR").unwrap()).join("gen_tests.rs");
    let mut outfile = File::create(outfile_path).unwrap();
    for file in test_files {
        let java_file = file.unwrap().to_str().unwrap().to_string();

        // FIXME: fill these in with your own logic for manipulating the filename.
        let name = java_file;
        let name1 = "NAME1";
        let name2 = "NAME2";

        write!(outfile, r#"
            #[test]
            fn test_globbed_{name}_null() {{
                check_files({name1}, {name2}, "null test");
            }}
            #[test]
            fn test_globbed_{name}_non_null() {{
                check_files({name1}, {name2}, "non-null test");
            }}
        "#, name=name, name1=name1, name2=name2).unwrap();
    }
}

In tests/tests.rs:

include!(concat!(env!("OUT_DIR"), "/gen_tests.rs"));