How do I read BASH parameters correctly from a file?

Question

I have this configuration file in my CI where I'm specifying a header file and some CMAKE flags on one line.

The configuration file looks like this (filelist):

./settings6.h -DMY_COMPILE_FLAGS="-m32 -fstrict-aliasing"
./settings7.h -DMY_FEATURE_1=ON
./settings8.h -DMY_FLAG=ON -DMY_FEATURE_2=ON -DMY_INCLUDE_DIR=/usr/include/

Now, I'm using a bash script to process this configuration file:

#!/bin/bash
SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
while read i; do
    HEADERFILE=$(echo $i | cut -d ' ' -f 1)
    CMAKEFLAGS=$(echo $i | cut -s -d ' ' -f 2-)
    if [[ "$HEADERFILE" == "" ]]; then
        continue
    fi
    CFLAGS="-Werror" cmake "my_build_dir" "$CMAKEFLAGS" -G "Ninja" -DMY_EXTRA_INCLUDE="$SCRIPTDIR/$HEADERFILE" -B"build_env_dir" > /dev/null
    ninja -C "build_env_dir"
done <<ENDOFINPUT
$(grep -v '^#' $SCRIPTDIR/filelist)
ENDOFINPUT

When I have the bash script as above, the line with settings6.h gets processed properly, i.e. the MY_COMPILE_FLAGS are set to -m32 -fstrict-aliasing.

However, settings8.h is failing because the value of MY_FLAG is seen by CMAKE as ON -DMY_FEATURE_2=ON -DMY_INCLUDE_DIR=/usr/include/, so MY_FEATURE_2 and MY_INCLUDE_DIR are not processed correctly.

After googling around a bit, I thought, well, surely a quoting issue, probably I have to remove the quotes around $CMAKEFLAGS like this:

    CFLAGS="-Werror" cmake "my_build_dir" $CMAKEFLAGS -G "Ninja" -DMY_EXTRA_INCLUDE="$SCRIPTDIR/$HEADERFILE" -B"build_env_dir" > /dev/null

In fact, this lets settings8.h work as expected (all three options are processed), but now, settings6.h is suddenly failing since CMAKE complains:

CMake Error: The source directory "/src/-fstrict-aliasing"" does not exist

Can someone guide me please how I read the settings correctly from my filelist so that settings6.h and settings8.h both succeed?

Possible duplicate of https://stackoverflow.com/questions/12136948/why-does-shell-ignore-quotes-in-arguments-passed-to-it-through-variables — tripleee, Nov 22 '19 at 10:53
@tripleee I actually don't think this is a duplicate: passing `$CMAKEFLAGS` as array doesn't improve anything. Do you have another suggestion? — eckes, Nov 22 '19 at 11:42
The best proposal I can come up with is to modify your input file format so that you *can* parse it without `eval` or other horrible workarounds. Wrapping this in a Makefile comes to mind as one potentially rather elegant workaround. — tripleee, Nov 22 '19 at 11:44
@tripleee do you have an example (maybe you post it as an answer) how such a file format would look like? — eckes, Nov 22 '19 at 13:26
If you are using `cmake` anyway I would probably use that for the whole thing, but I'm not familiar enough with it to show you how. — tripleee, Nov 22 '19 at 14:28
Possible duplicate of https://stackoverflow.com/questions/4227994/how-do-i-use-the-lines-of-a-file-as-arguments-of-a-command — tripleee, Nov 22 '19 at 16:47
@tripleee I finally went for doing it all with CMAKE. Thanks for poining me there! — eckes, Nov 25 '19 at 12:40

tripleee · Answer 1 · 2019-11-23T11:21:03.140

1

Here's a Makefile which refactors this into a sequence of recipes.

Cases := $(patsubst %.h,%,$(wildcard ./settings*.h))
all_done := $(patsubst %,.%.done,$(Cases))

.PHONY: all
all: $(all_done)

cases.mk: filelist.txt
    sed 's%^\./%case_%;s% % := %' $< >$@
include cases.mk

.%_done: ./%.h
    CFLAGS="-Werror" cmake "my_build_dir" $(case_$*) -G "Ninja" \
        -DMY_EXTRA_INCLUDE="$<" -B"build_env_dir" > /dev/null
    ninja -C "build_env_dir"

edited Nov 23 '19 at 11:21

answered Nov 23 '19 at 08:54

tripleee

175,061
34
275
318

Not in a place where I can test this but hopefully this can at least inspire some lateral thinking... – tripleee Nov 23 '19 at 08:57

How do I read BASH parameters correctly from a file?

1 Answers1