cmake generate test data

Question

my tests require some data files to be present before they can run. The files are huge so I would not like to commit them into my VCS but instead generate them on demand using a script. But generating the files also takes a lot of time, so I'd like to only generate them only once and not again every time I run make test.

Is it possible to let my test depend on the presence of a file on disk and tell cmake to regenerate that file if it is either not present or out of date?

I tried the following but it doesn't seem to work:

enable_testing()
add_test(NAME foo COMMAND bla myfile.txt)
set_tests_properties(foo PROPERTIES DEPENDS myfile.txt)
add_custom_command(OUTPUT myfile.txt COMMAND createmyfile)

Is it possible at all?

You could take either add it as a post-build step to building `bla` or you could add `CTEST_CUSTOM_PRE_TEST` command globally or to `CTestCustom.cmake` (see [Customizing CTest](https://cmake.org/Wiki/CMake/Testing_With_CTest#Customizing_CTest)). — Florian, Mar 15 '17 at 11:22
@Florian but I don't want a post-build step - I want to specify a prerequisite of the test. Otherwise I cannot just run "make test" and assume everything that is needed has already been compiled (or will be compiled). And I don't see how `CTEST_CUSTOM_PRE_TEST` could work because that command is *always* executed and not only when needed. — josch, Mar 15 '17 at 11:42
Explanations for duplicate: You want to **generate some file** for the test. Duplicate questions asks about **building test executable** for the test. From the view of CMake both actions are just CMake targets. — Tsyvarev, Mar 15 '17 at 13:55
@Tsyvarev thanks, that question is indeed a duplicate of mine even though sadly, it does not offer a satisfying solution. :( — josch, Mar 15 '17 at 14:41
You may combine approach in the [second answer](http://stackoverflow.com/a/10824578/3440745) with *CTEST_CUSTOM_PRE_TEST* suggested by @Florian: As a prepation to the tests you may ask CMake to build specific target via `cmake --build <...> --target=<...>` command. — Tsyvarev, Mar 15 '17 at 18:51
@Tsyvarev but `--target` only allows to specify a single target. This becomes more and more convoluted with more depenencies per test. — josch, Mar 16 '17 at 13:08
Hm, you may create "pure" target `add_custom_target(prepare_tests)` and use that target with `--target` option for *CTEST_CUSTOM_PRE_TEST* command. For all targets *t*, required to be built for tests, use `add_dependencies(prepare_tests t)`. This looks like as an emulation of `test` CMake target (CMake doesn't allow to use this target directly). — Tsyvarev, Mar 16 '17 at 13:44
@Tsyvarev That would get even more messy. Different tests have different prerequisits. What if I only want to run some of the tests? What if the prerequisits have dependencies between each other? As can be seen in the question that this question was marked a duplicate of, there is no good answer because this just cannot be done with cmake right now. — josch, Mar 16 '17 at 13:54
From your question I understand, that you want to build the file on `make test`. With this command you *cannot select tests* for run. If you want **per-test** preparations ... at least, this would be a separate question. — Tsyvarev, Mar 16 '17 at 14:12
@Tsyvarev That doesn't invalidate the concern about prerequisits between each other. — josch, Mar 16 '17 at 14:46
If prerequisit `p1` depends on `p2`, then you have something like `add_dependencies(p1 p2)`. In that case `cmake --build <...> --target=p1` will build both `p1` and `p2` in correct order. Do I understand your last comment correctly? Otherwise, can you provide some example (in the question post) of what you want to achive? — Tsyvarev, Mar 16 '17 at 19:07

Th. Thielemann · Answer 1 · 2017-03-16T18:23:48.337

0

Yes it is possible at all. CMake can add dependencies between targets and files and those files needs not to be source files.

Your code steps looks good (I didn't build it) but I assume what is missing is the build of the test.

And I use add_custom_target instead of add_custom_command.

CMakeLists.txt

cmake_minimum_required(VERSION 2.8.12)
add_library(mylib SHARED mylib.cpp)

# Create out.dat as an example 
add_custom_target(testfile_outdat COMMAND touch out.dat)

add_executable(mytest mytest.cpp)
target_link_libraries(mytest mylib)
add_dependencies(mytest testfile_outdat)
add_test(NAME mytest COMMAND mytest)

This example generates the code while make is executed. This solution fits in case the data are not modified by tests.

The test data creation depends on made of target mytest, but not of the declaration that mytest is to be used as test

Depending on the commands to create the test data file, some platform specific stuff could be possible. Thus you can create different custom_targets depending on the platform to support.

mylib.h

bool compare(int a, int b);

mylib.cpp

#include "mylib.h"
bool compare(int a, int b)
{
    return a == b;
}

mytest.cpp

#include <stdio.h>
#include "mylib.h"
int main(int arcc, char* argv[])
{
    bool actual = compare(1, 1);
    bool expected = true;
    printf("Testresult %s\n", (actual == expected) ? "OK" : "Failed");
    return 0;
}

edited Mar 16 '17 at 18:23

answered Mar 15 '17 at 12:57

Th. Thielemann

2,592
1
23
38

Great. How? I already explained what I tried and that it doesn't work. – josch Mar 15 '17 at 12:58
I extended the example. But "touch" is platform-specific. – Th. Thielemann Mar 15 '17 at 13:10
You assume that my problem is, that I don't know how to build the executable doing the test but that is not the problem (and who says that the executable has to be compiled in the first place and is not just a shell script?). My problem is, that I want my test to depend on an input file that is generated on demand (and not regenerated if not needed). Please tell me what is unclear about my question so that I can clear up the confusion. – josch Mar 15 '17 at 13:23
But this is done with the example code. Which part is unclear and has to rework? – Th. Thielemann Mar 15 '17 at 13:50
Your example code never uses `add_test`. If I use `add_dependencies` with a target name used in an `add_test` statement I just get the error `Cannot add target-level dependencies to non-existent target`. The error message even informs me that `add_dependencies` only works with top-level targets created by `add_executable`, `add_library`, or `add_custom_target`. It doesn't work with `add_test`. – josch Mar 15 '17 at 14:30
Example extended by add_test – Th. Thielemann Mar 15 '17 at 14:52
I cannot confirm this to work. Did you try it out before posting? That you only added `add_test` later makes me fear that you didn't actually try it. This is also what the bug report mentioned in the answers to the question that this question is marked a duplicate as suggests. You cannot create dependencies of tests on other targets with current cmake. Also, `add_custom_target` creates a target that is always considered out of date which is not what I want either. – josch Mar 16 '17 at 13:11
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/138234/discussion-between-th-thielemann-and-josch). – Th. Thielemann Mar 16 '17 at 14:29

cmake generate test data

1 Answers1