5

I have this strange issue with my bash script. I compile boost as part of it. The call from the script looks like this:

./b2 --reconfigure ${PARALLEL} link=static cxxflags=-fPIC install boost.locale.iconv=off boost.locale.posix=off -sICU_PATH="${ICU_PREFIX}" -sICU_LINK="${BOOST_ICU_LIBS}" >> "${BOOST_LOG}" 2>&1

That command works perfectly well. The log file shows that it finds ICU without a problem. However, if I change it to run from a variable, it no longer finds ICU (but it still compiles everything else):

bcmd="./b2 --reconfigure ${PARALLEL} link=static cxxflags=-fPIC install boost.locale.iconv=off boost.locale.posix=off -sICU_PATH=\"${ICU_PREFIX}\" -sICU_LINK=\"${BOOST_ICU_LIBS}\""
$bcmd >> "${BOOST_LOG}" 2>&1

What's the difference? I would like to be able to use the second approach so that I can pass the command into another function before running it.

Brannon
  • 5,324
  • 4
  • 35
  • 83
  • 1
    Do note that using `eval` is dangerous. See [Eval command and security issues](https://mywiki.wooledge.org/BashFAQ/048) – Inian Nov 26 '18 at 08:30

3 Answers3

3

Don't use a variable to store complex commands involving quotes that are nested. The problem is when you call the variable with just $cmd, the quotes are stripped incorrectly. Putting commands (or parts of commands) into variables and then getting them back out intact is complicated.

Quote removal is part of the one of the word expansions done by the shell. From the excerpt seen in POSIX specification of shell

2.6.7 Quote Removal

The quote characters ( backslash, single-quote, and double-quote) that were present in the original word shall be removed unless they have themselves been quoted.

Your example can be simply reproduced by a simple example. Assuming you have a few command flags (not actual ones)

cmdFlags='--archive --exclude="foo bar.txt"'

If you carefully look through the above, it contains 2 args, one --archive and another for --exclude="foo bar.txt", notice the double-quotes which needs to be preserved when you are passing it.

Notice how the quotes are incorrectly split when I don't quote cmdFlags, in the printf() call below

printf "'%s' " $cmdFlags; printf '\n'
'--archive' '--exclude="foo' 'bar.txt"'

and compare the result with one with proper quoting done below.

printf "'%s' " "$cmdFlags"; printf '\n'
'--archive --exclude="foo bar.txt"'

So along with the suggestion of properly quoting the variable, the general suggestion would be to use an array to store the flags and pass the quoted array expansion

cmdArray=()
cmdArray=(./b2 --reconfigure ${PARALLEL} link=static cxxflags=-fPIC install boost.locale.iconv=off boost.locale.posix=off -sICU_PATH="${ICU_PREFIX}" -sICU_LINK="${BOOST_ICU_LIBS}")

and pass the array as

"${cmdArrray[@]}" >> "${BOOST_LOG}" 2>&1
Community
  • 1
  • 1
Inian
  • 80,270
  • 14
  • 142
  • 161
  • 1
    It's exactly what I do. When the command line becomes complex and contains quoted strings with whitespaces or other special characters, I use arrays. My only comment is the fact that the `cmdArray=()` is redundant. The assignment on the following line will make bash create the `cmdArray` variable directly as an array. The only situation when you need to specifically define the variable type is when using associative arrays. But indexed ones are automatically declared as such. – alindobre Nov 24 '18 at 07:41
0

Try to use eval when you want to execute a string as a command. This way, you won't have issues regarding strings that have spaces etc. The expanded cmd string is not re-evaluated by bash hence, things like "hi there" are expanded as two separate tokens.

eval "$bcmd" >> "${BOOST_LOG}" 2>&1

To demonstrate this behavior, consider this code:

cmd='echo "hi there"'
$cmd
eval "$cmd"

Which outputs to:

"hi there"
hi there

The token "hi there" is not re-evaluated as a quoted string.

ssemilla
  • 3,900
  • 12
  • 28
0

Use single quota instead of two.

bcmd='./b2 --reconfigure ${PARALLEL} link=static cxxflags=-fPIC install boost.locale.iconv=off boost.locale.posix=off -sICU_PATH=\"${ICU_PREFIX}\" -sICU_LINK=\"${BOOST_ICU_LIBS}\"'

Bash documentation states that single quota does not interpolate:

3.1.2.2 Single Quotes

Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

That way you omit double-quota stripping and removing problem, and your string will be passed correctly. If you want to stay with double quotes, you have to vary that they DO NOT preserve literal value of certain characters: $, ' and \ unless preceded with \, as manual states:

3.1.2.3 Double Quotes

Enclosing characters in double quotes (") preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash.

In your example you forgot to mark $ with backslash as well. Difference between these two is explained perfectly by Adam here: Differences between single and double quota

PStarczewski
  • 479
  • 2
  • 17