Do not expand CMake list variable

Question

I have a CMake script that runs some tests via add_test(), running under Windows (Server 2008, don't ask) in CMake 3.15. When these tests are called, the PYTHONPATH environment variable in the environment they run in seems to get reset to the environment default, and doesn't contain some paths that it needs to.

I therefore need to set PYTHONPATH when the tests are run to the value of the $ENV{PYTHONPATH} variable when CMake runs. This has a number of semicolon-separated paths, so CMake thinks it's a list and tries to expand it into a number of space-separated strings, which obviously ends badly.

I cannot work out how to stop CMake doing this. From everything I can see, you should be able to do just surround with quotes:

add_test(
  NAME mytest
  COMMAND cmake -E env PYTHONPATH="$ENV{PYTHONPATH}"
  run_test_here)

...but it always does the expansion. I also tried setting with set_tests_properties:

set_tests_properties(mytest PROPERTIES
    ENVIRONMENT PYTHONPATH="$ENV{PYTHONPATH}")

...but that didn't appear to do anything at all - PYTHONPATH at test time wasn't altered. I thought it was because it's an environment variable, but using a regular CMake variable via set() makes no difference, so I'm doing something wrong. Help please!

I want: ```cmake COMMAND cmake -E env PYTHONPATH="C:\part\of\pythonpath;C:\another\part" ``` ...but not hardcoded, obviously. — Ben Taylor, Nov 01 '19 at 10:47
Shouldn't PYTHONPATH be separated by `:`? It shouldn't be semicolon separated paths, it should be colon separated paths. What is the output of `COMMAND echo "$ENV{PYTHONPATH}"` and `message(STATUS "ENV{PYTHONPATH} = $ENV{PYTHONPATH}")` before running the command? — KamilCuk, Nov 01 '19 at 10:54
On Linux that'd be true. Sadly I'm stuck with Windows, which uses semicolons. — Ben Taylor, Nov 01 '19 at 10:56
`message(STATUS "ENV{PYTHONPATH} = $ENV{PYTHONPATH}")` returns what it should (colon-separated list of paths, which looks like `C:\Python27\Scripts;E:\JenkinsMIDEBLD\workspace\...;...`. `COMMAND echo "$ENV{PYTHONPATH}"` returns the same thing but with spaces: `"C:\Python27\Scripts" "E:\JenkinsMIDEBLD\workspace\..." "..."` — Ben Taylor, Nov 01 '19 at 10:59

KamilCuk · Accepted Answer · 2019-11-01T11:18:46.873

The following should work:

COMMAND cmake -E env "PYTHONPATH=$ENV{PYTHONPATH}"

You need to quote the full part of the command line, to make properly expanded message.

Tested with:

set(tmp "C:\\Python27\\Scripts;E:\\JenkinsMIDEBLD\\workspace\\...;...")
add_test(NAME MYtest1 COMMAND cmake -S . -E env "tmp=${tmp}") 
add_test(NAME MYtest2 COMMAND cmake -S . -E env tmp="${tmp}")

After running ctest I get:

1: Test command: /bin/cmake "-S" "." "-E" "env" "tmp=C:\Python27\Scripts;E:\JenkinsMIDEBLD\workspace\...;..."
2: Test command: /bin/cmake "-S" "." "-E" "env" "tmp="C:\Python27\Scripts" "E:\JenkinsMIDEBLD\workspace\..." "...""

The first test has proper ; passed to var, while the second one passes space separated list.

This is how cmake parses quoted arguments. An argument is either fully quoted or not quoted at all - partial quotes are interpreted as a literal ". So assumnig that:

set(var a;b;c)

The following:

var="$var"

Is not a quoted argument and " are taken literally! It expands the $var list into space separated list and the " stay, there is one " between = and a, and there is additional " on the end. The var="$var" is equal to:

var=\"a b c\"
    ^^     ^^    - the quotes stay!
^^^^^^^ ^ ^^^    - these are 3 arguments, the last one is `c"`

Without quotes is:

var=$var

is equal to (notice the missing quotes):

var=a c c

To quotes argument you have to quote it all, with first and last character of the element beeing ":

"var=$var"

will expand to:

"var=a;b;c"

Hurray, it works! The one configuration of quotes and things I didn't try. Many thanks! — Ben Taylor, Nov 01 '19 at 12:07

score 0 · Answer 2 · answered Feb 12 '21 at 20:43

You can make this work with the ENVIRONMENT test property, but there's a catch:

Semicolon separates different environment variables to set; you need to escape semicolons in your environment variable. For example instead of

set_tests_properties(mytest PROPERTIES
    ENVIRONMENT "PYTHONPATH=foo;bar")

you need to use

set_tests_properties(mytest PROPERTIES
    ENVIRONMENT "PYTHONPATH=foo\\;bar")

The fact that an environment variable may contain semicolons makes some transformation necessary: since ; is used to separate list elements you can simply use list(JOIN) to replace those with "\\;".

The following example works with PATH, not PYTHONPATH, since I don't have python installed:

CMakeLists.txt

cmake_minimum_required(VERSION 3.12.4) # required for list(JOIN)

project(TestProject)

# get a version of the PATH environment var that can be used in the ENVIRONMENT test property
set(_PATH $ENV{PATH})
list(JOIN _PATH "\\;" _PATH_CLEAN)

# just use a cmake script so we don't need to require any program able to retrieve environment vars
add_test(NAME Test1 COMMAND "${CMAKE_COMMAND}" -P "${CMAKE_CURRENT_SOURCE_DIR}/test_script.cmake")

# we add another simpler var to test in the cmake script
set_tests_properties(Test1 PROPERTIES ENVIRONMENT "PATH=${_PATH_CLEAN};FOO=foo")

enable_testing()

test_script.cmake

message(STATUS "PATH=$ENV{PATH}")

if (NOT "$ENV{FOO}" STREQUAL "foo")
    # the following command results in a non-0 exit code, if executed
    message(FATAL_ERROR "FOO environment var should contain \"foo\" but contains \"$ENV{FOO}\"")
endif()

Do not expand CMake list variable

2 Answers2

Linked