Multi-pass variable expansion, is it possible?

Question

If you have a bash variable that contains other variables, is it possible to "double" expand it as part of a command?

For example:

# Example #1
# ----------
# We define our first variable
TOP_LEVEL=/usr;

# Then use it (trivially) to set the values of other variables when executing a command
INC_DIRS="-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include" \
LIB_DIRS="-L$TOP_LEVEL/lib -L$TOP_LEVEL/local/lib" \
./my_command

This is fairly self-explanatory. Now let's expand this example to the problem scenario:

# Example #2
# ----------
# We define our first variable
TOP_LEVEL=/usr;

# Notice the use of single quotes below, to defer expansion of `TOP_LEVEL` to some point
# later - the time when DEFERRED_EVAL_INC_DIRS will actually be used
DEFERRED_EVAL_INC_DIRS='-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include'
... # Other stuff happens here that may change the value of TOP_LEVEL, e.g.:
TOP_LEVEL=/usr/confused
# Since we used single quotes to define DEFERRED_EVAL_INC_DIRS, it is not affected

# Now we want to execute a command like before, and pass it three variables,
# INC_DIRS, LIB_DIRS, and FOO_LIB_INC_DIRS, in their fully expanded form:
INC_DIRS="-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include" \
LIB_DIRS="-L$TOP_LEVEL/lib -L$TOP_LEVEL/local/lib" \
FOO_LIB_INC_DIRS="$DEFERRED_EVAL_INC_DIRS" ./my_command

The expansion of INC_DIRS and LIB_DIRS is trivial, they become:

INC_DIRS='-I/usr/confused/include -I/usr/confused/local/include'
LIB_DIRS='-L/usr/confused/lib -I/usr/confused/local/lib'

The problematic expansion is with FOO_LIB_INC_DIRS. The desired (fully expanded value should be):

FOO_LIB_INC_DIRS='-I/usr/confused/include -I/usr/confused/local/include'

..., finally expanding the embedded $TOP_LEVEL variables used in the value of DEFERRED_EVAL_INC_DIRS to their present value of /usr/confused. So, there is a second evaluation pass needed for that variable and it has spaces in its value that need to be preserved as well, just to make life even more difficult.

Had DEFERRED_EVAL_INC_DIRS referred to a single variable by name (and contained no other characters), the ${!DEFERRED_EVAL_INC_DIRS} indirect variable reference construct could have been used. But, this is a more complicated scenario with surrounding text and multiple uses of the variable that needs to be expanded.

I tried playing with bash's eval, but could not get this expansion to happen in one command line and without introducing additional variables, basically as an expansion of the code segment above. The full desired expanded result is:

INC_DIRS='-I/usr/confused/include -I/usr/confused/local/include' \
LIB_DIRS='-L/usr/confused/lib -I/usr/confused/local/lib' \
FOO_LIB_INC_DIRS='-I/usr/confused/include -I/usr/confused/local/include' \
./my_command

Is there a simple change or set of changes that can be made to the command invocation in Example #2 in order for the full expansion to occur?

Update: This is neither a duplicate (at least in any obvious way) of similar questions, due to the more complex deferred expansion involved, while preserving word boundaries across spaces. If the solution seems obvious to you, please post it as an answer and a change to:

INC_DIRS="-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include" \
LIB_DIRS="-L$TOP_LEVEL/lib -L$TOP_LEVEL/local/lib" \
FOO_LIB_INC_DIRS="$DEFERRED_EVAL_INC_DIRS" ./my_command

..., to expand the variable DEFERRED_EVAL_INC_DIRS fully, respecting word boundaries properly, and producing:

INC_DIRS='-I/usr/confused/include -I/usr/confused/local/include' \
LIB_DIRS='-L/usr/confused/lib -I/usr/confused/local/lib' \
FOO_LIB_INC_DIRS='-I/usr/confused/include -I/usr/confused/local/include' \
./my_command

..., preferably without creating a subshell if possible (and without breaking up this command into multiple commands or adding temporary variables, which would make this a much simpler case).

I have (unsuccessfully) tried using `eval` in-line. Also, this is not a duplicate - my example is far more complicated than the linked question/answer, because of the embedded spaces and references to the variable multiple times. — Michael Goldshteyn, Jul 30 '18 at 03:18
"far more complicated" does not mean that it isn't the same question. — Mad Physicist, Jul 30 '18 at 03:19
It also does not mean that the answer given to the other question applies here in an "obvious" manner. — Michael Goldshteyn, Jul 30 '18 at 03:20
@MichaelGoldshteyn I guess you already knew the answer I added below? Why can't you just defer the `eval` if `$TOP_LEVEL` could change? — Alex Harvey, Jul 30 '18 at 03:22
@Alex, because the deferred variable acts as a sort of *template* that is created up front, but should only get expanded at the time of use. Also, it is used multiple times and should not have to be defined multiple times. — Michael Goldshteyn, Jul 30 '18 at 03:33
Even so I still don't understand why the entire template can't be run via eval, e.g. in the same way you do with `eval \`ssh-agent\`` for instance. — Alex Harvey, Jul 30 '18 at 03:35
@Alex, if you think that will result in a solution that solves the question asked, could you please modify your answer accordingly and I will checkmark it? — Michael Goldshteyn, Jul 30 '18 at 03:42

Alex Harvey · Answer 1 · 2018-07-30T03:56:15.770

1

You can use eval here but you need to call it when you set the "lower level" vars, e.g. try this:

TOP_LEVEL="/usr"
eval "INC_DIRS='-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include'"
echo TOP_LEVEL: $TOP_LEVEL
echo INC_DIRS: $INC_DIRS

And of course be aware of the dangers of using eval e.g. here.

Also, as mentioned in the followup comments, you could also treat the "template" as a file that is to be evaled, e.g. try this:

$ TOP_LEVEL="/usr"
$ cat template.sh 
INC_DIRS='-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include'
$ eval $(<template.sh)
$ echo $INC_DIRS 
-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include

edited Jul 30 '18 at 03:56

answered Jul 30 '18 at 03:17

Alex Harvey

14,494
5
61
97

Is there a way to embed that eval into the command itself? The point was to define a variable to refer to another with expansion deferred to the point of use of the referring variable. – Michael Goldshteyn Jul 30 '18 at 03:26
I don't think so. Perhaps someone else can say for certain. – Alex Harvey Jul 30 '18 at 03:27
That is the crux of my question. Insert the line `TOP_LEVEL='/usr/confused'` before the `echo`s and after the `eval`. `INC_DIRS` (incorrectly) retains the old value of `TOP_LEVEL`, '/usr', because expansion was immediate (i.e., not deferred to the point of use in the last line). – Michael Goldshteyn Jul 30 '18 at 03:31
Updated as discussed above. – Alex Harvey Jul 30 '18 at 03:57
What you are echoing does not have `$TOP_LEVEL` expanding as it should at the point of the use of `$INC_DIRS`. Also, I would like to see this as part of the command being executed, specifically: `INC_DIRS="-I$TOP_LEVEL/include -I$TOP_LEVEL/local/include" \ LIB_DIRS="-L$TOP_LEVEL/lib -L$TOP_LEVEL/local/lib" \ FOO_LIB_INC_DIRS="$DEFERRED_EVAL_INC_DIRS" ./my_command`. Change the `"$DEFERRED_EVAL_INC_DIRS"` to what you think it should be so that proper expansion occurs. – Michael Goldshteyn Jul 30 '18 at 15:38
It seems to me that you're trying to do something like multiple indirection in Bash like you can do in C++ and I believe it's not possible. You could try asking the question again. The way it's presented above gives the impression that it's about eval and of course it's not really. I'm curious to know too now. – Alex Harvey Jul 31 '18 at 13:43

Multi-pass variable expansion, is it possible?

1 Answers1