23

I'm building my project with CMake, and I'm trying to create a bunch of test suites for each module. Apparently if I modify the variable CMAKE_RUNTIME_OUTPUT_DIRECTORY then ctest cannot find the test to run and fails.

I've made a minimal example to showcase what I am talking about, and I'm running it with CMake 2.8.11.2 on Lubuntu 13.10. I'd appreciate if somebody could tell me whether this is a bug and/or how to work around it. Thanks.

file CMakeLists.txt:

cmake_minimum_required (VERSION 2.6)
project (Test)

# Put all tests in the test directory, where the sources also are
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR}/test)

enable_testing()

add_subdirectory (${PROJECT_SOURCE_DIR}/test)

file test/CMakeLists.txt:

cmake_minimum_required(VERSION 2.6)

add_executable(ttest main.cpp)
add_test(ttest ttest)

file test/main.cpp:

int main() {
    return 0;
}

After building in a new folder build, the executable is correctly created in the folder test. Running make test from build results in the following output:

Running tests...
Test project /home/svalorzen/Tests/cmake/build
    Start 1: ttest
Could not find executable ttest
Looked in the following places:
ttest
ttest
Release/ttest
Release/ttest
Debug/ttest
Debug/ttest
MinSizeRel/ttest
MinSizeRel/ttest
RelWithDebInfo/ttest
RelWithDebInfo/ttest
Deployment/ttest
Deployment/ttest
Development/ttest
Development/ttest
Unable to find executable: ttest
1/1 Test #1: ttest ............................***Not Run   0.00 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.00 sec

The following tests FAILED:
      1 - ttest (Not Run)
Errors while running CTest
make: *** [test] Error 8
Svalorzen
  • 5,353
  • 3
  • 30
  • 54
  • If running with [`ctest`](https://cmake.org/cmake/help/latest/manual/ctest.1.html#options), you can specify a command line option to tell CTest to look for the tests in a different location: `ctest --test-dir /path/to/tests`. Note, you must use CMake 3.20 or greater to get this CLI feature. – Kevin Feb 19 '21 at 16:13

4 Answers4

27

edit :

actually I missed something in the documentation for add_test:

If COMMAND specifies an executable target (created by add_executable) it will automatically be replaced by the location of the executable created at build time

So using ttest instead of $<TARGET_FILE:ttest> should work.

I got the same problem but I do not really know if it is a bug or not.

The solution I found for this is providing the path of the test executable in the command (with the target ttest):

in test/CMakeLists.txt :

add_test(ttest test/ttest)

Eventually, you would like to store the path in a variable in order to write something like ${test_dir}/ttest.

If you want something more robust, the best is to use the long add_test command and generator expressions :

add_test(NAME ttest COMMAND $<TARGET_FILE:ttest>)

The generation expression $<TARGET_FILE:ttest> will expand to the output file of the executable target (ttest here), so you are sure there is no problem with the directory. There is other expression referenced in cmake documentation.

It can get a little verbose if you have lot of tests to declare so I use a macro like :

macro (create_test target)
  add_test (NAME ${target} COMMAND $<TARGET_FILE:${target}>)
endmacro (create_test)

#[some code]

#test definition
create_test(ttest)

Assuming the test name is the same as the executable name.

I could not find any other working solution. It is possible to set the working directory with add_test, which apparently make ctest find the executable but the test then crashes with a "BAD_COMMAND" error.

I'm new with cmake, so may be there is another solution.

starball
  • 20,030
  • 7
  • 43
  • 238
archz
  • 1,073
  • 13
  • 19
  • 4
    I also had the problem that my tests needed some data that was in the folder and was not being found, given that the working directory was still kept wrong. In case somebody is reading this, you can fix this using `WORKING_DIRECTORY ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}` before the `COMMAND` clause. – Svalorzen Jan 29 '14 at 11:04
  • A fullpath to the executable did not work for me e.g. /home/.../bin/mytest. It appears to append the add_test argument to ${PROJECT_SOURCE_DIR} e.g. for add_test(_mytest bin/mytest), the path becomes ${PROJECT_SOURCE_DIR}/bin/mytest . – JayS Apr 20 '17 at 17:10
  • Any chance you'd be able to answer [this followup](https://stackoverflow.com/q/48014060/2069064) three years later? – Barry Dec 28 '17 at 20:38
  • Very important not to put a space after TARGET_FILE: – Pete Jun 04 '20 at 14:45
7

Use command:

add_test(NAME ttest COMMAND $<TARGET_FILE:ttest>)

Writing NAME and COMMAND is mandatory when using TARGET_FILE (or at least it's not working for me otherwise. Use cmake --help-command add_test for more information.

silgon
  • 6,890
  • 7
  • 46
  • 67
2

You will also have this problem if your test command contains parameters and you quote them together with the executable name.

So if you have something like this:

    add_test(NAME "name" COMMAND "testExe arg1 arg2 arg3")

Instead do this:

    add_test(NAME "name" COMMAND "testExe" "arg1" "arg2" "arg3")

Otherwise CMake will try to find a test executable name with spaces in it.

MatrixManAtYrService
  • 8,023
  • 1
  • 50
  • 61
1

Using clearly NAME and COMMAND fixes problem for me

add_test(NAME ttest COMMAND ttest)
Володимир Ніколайчук
  • 603
  • 6
  • 4