4

Background

I am writing integration tests using the guidelines in the Rust Book. A requirement is to be able to run setup and teardown code at two levels:

  1. Test level: Before and after each test.
  2. Crate level: Before and after all tests in the crate.

The basic idea is as follows:

  1. I wrote a TestRunner class.
  2. Each integration test file (crate) declares a lazy-static singleton instance.
  3. Each #[test] annotated method passes a test function to a run_test() method of the TestRunner.
  4. With this design it is straightforward to implement Test level setup and teardown. The run_test() method handles that.
  5. It is also no problem to implement the setup part of the Crate level, because the TestRunner knows when run_test() is called for the first time.

The Problem

The remaining problem is how to get the TestRunner to execute the Crate level teardown after the last test has run. As tests may run in any order (or even in parallel), the TestRunner cannot know when it is running the last test.

What I Have Tried

I used the ctor crate and the #[dtor] annotation to mark a function as one that will run before the process ends. This function calls the Crate level teardown function. This is an unsatisfactory solution because of this issue which documents the limitations of what can be done in a dtor function.

A Step in the Right Direction

I propose to pass a test_count argument to the TestRunner's constructor function. As the TestRunner now knows how many test functions there are, it can call the Crate level teardown function after the last one completes. (there are some thread safety issues to handle, but they are manageable).

The Missing Link

Clearly, the above approach is error prone as it depends on the developer updating the test_count argument every time she adds or removes a test or marks one as ignored.

The Remaining Problem

I would therefore like to be able to detect the number of tests in the crate at compile time without any manual intervention by the developer.

I am not familiar enough with Rust macros to write such a macro myself, but I assume it is possible.

Maybe it is even possible to use the test crate object model for this (See https://doc.rust-lang.org/test/struct.TestDesc.html)

Can anyone suggest a clean way to detect (at compile time) the number of tests that will run in the crate so I can pass it to the constructor of the TestRunner?

Herohtar
  • 5,347
  • 4
  • 31
  • 41
David Sackstein
  • 500
  • 1
  • 5
  • 19
  • I don’t see why you can’t simply run a clean function after the test runners finish running test. – Netwave Oct 10 '21 at 22:03
  • How do I know (indeed, how does the test runner know) that it has finished running the last test? – David Sackstein Oct 10 '21 at 23:07
  • IIUC, you can't know the number of tests at compile time because the test framework may run only a subset of the tests depending on [command-line arguments](https://doc.rust-lang.org/book/ch11-02-running-tests.html#running-single-tests). – Jmb Oct 11 '21 at 06:44
  • 1
    Does this answer your question? [Is there a way to execute a teardown function after all tests have been run?](https://stackoverflow.com/questions/59077877/is-there-a-way-to-execute-a-teardown-function-after-all-tests-have-been-run) – Jmb Oct 11 '21 at 07:11
  • @Jmb Yes, you are right. So can I know at runtime? – David Sackstein Oct 11 '21 at 19:01
  • @Jmb Thanks for the pointer. This is the same question, but there is no answer. Which is also an answer, I guess. I find it incredible that a modern language such as Rust doesn't have a simple out of the box way to meet such requirements. It seems, that not only is there no out of the box solution, but the ecosystem just won't let you even write one yourself. Oh, well. – David Sackstein Oct 11 '21 at 19:04

0 Answers0