0

I was exploring different methods of doing indirect variable access in Bash (4.4.19 on macOS via MacPorts) when I stumbled upon this inconsistency:

Why does the second loop below only loop once when IFS is set to ':'?

Note: the impact of IFS is notable here!

#!/usr/bin/env bash
PATH="foo:bar:baz"
var=PATH
IFS=':'

echo ${!var}
# returns foo bar baz
echo $(eval echo \$$var)
# also returns foo bar baz

for V in ${!var}; do
    echo $V
done
# returns foo\nbar\nbaz\n

for V in $(eval echo \$$var); do
    echo $V
done
# returns foo bar baz

If you replace the PATH contents with space-separated items and removed IFS, it works as expected in both cases.

John de Largentaye
  • 893
  • 4
  • 19
  • Some of these cases have potential security bugs if given invalid inputs. Some don't. – Charles Duffy May 25 '18 at 02:02
  • 1
    Compare behavior when `var='PATH$(touch /tmp/evil)'` -- some versions (particularly, the `eval`-based ones) will create `/tmp/evil` (and could instead of deleted your home directory or done any other thing an attacker able to set `var` could have selected). Others will throw an error. Which is safer, in that context, is quite clear. – Charles Duffy May 25 '18 at 02:03
  • 1
    BTW, there are a whole lot of quoting-based bugs here. Running your code through http://shellcheck.net/ would find them; see also [BashPitfalls #14](http://mywiki.wooledge.org/BashPitfalls#echo_.24foo). – Charles Duffy May 25 '18 at 02:04
  • You're focusing on the security bugs related to eval and no quoting. I understand that they're worth pointing out, but I don't see how they actually relate, in this case, to the behavior difference. – John de Largentaye May 25 '18 at 02:58
  • To explain differently: Note that `$(eval echo \$$var)` will string-split the results of the `$PATH` expansion, so the `echo` inside the command substitution splits arguments with spaces. Spaces aren't present inside `IFS`, so the `for` loop doesn't split on them, so it iterates only once. – Charles Duffy May 25 '18 at 16:05
  • 1
    ...by contrast, in `for V in ${!var}`, there is no `echo` converting your `:`s to spaces, so they're still colons when they get string-split into pieces for the `for` loop to iterate over, so the loop executes multiple times. – Charles Duffy May 25 '18 at 16:06
  • 1
    But if you changed `for V in $(eval echo \$$var); do` to `for V in $(eval "echo \"\$$var\""); do`, the behavior is identical to the other form, but for the extra security bugs. What you asked about here is **purely** a quoting difference, nothing else. – Charles Duffy May 25 '18 at 16:07

1 Answers1

1

Code-Specific Demonstration

IFS=:
var=path
path=foo:bar:baz

printf '%s\n' "Example 1: Eval Running Unquoted Echo"
printf ' - %s\n' $(eval echo \$$var)

printf '%s\n' '' "Example 2: Eval Running Quoted Echo"
printf ' - %s\n' $(eval "echo \"\$$var\"")

printf '%s\n' '' "Example 3: Indirect Expansion Syntax With Unquoted Echo"
printf ' - %s\n' $(echo ${!var})

printf '%s\n' '' "Example 4: Indirect Expansion Syntax Without Unquoted Expansions"
printf ' - %s\n' ${!var}

...with the output:

Example 1: Eval Running Unquoted Echo
 - foo bar baz

Example 2: Eval Running Quoted Echo
 - foo
 - bar
 - baz

Example 3: Indirect Expansion Syntax With Unquoted Echo
 - foo bar baz

Example 4: Indirect Expansion Syntax Without Unquoted Expansions
 - foo
 - bar
 - baz

The difference between the working examples and the broken examples has nothing whatsoever to do with whether eval is used; it has only anything to do with whether there is an echo subprocess without quotes.

Generalized Answer

Explaining The Difference: Unquoted Command Substitutions

The difference in your output between the cases given is nothing to do with the indirection, and everything to do with the unquoted expansions; see BashPitfalls #14.

Consider the following example, which uses no indirection at all:

var='foo
bar
baz'

echo "Correct version:"
echo "$var"
echo
echo "Incorrect version:"
echo $var

...its output is:

Correct version:
foo
bar
baz

Incorrect version:
foo bar baz

There's no indirection at all here -- the only difference is that between echo $foo and echo "$foo".

In exactly the same way, echo $(...) removes your newlines, where echo "$(...)" would retain them.

So Why Are There Multiple Command Substitution Mechanisms?

Because the ones using eval are insecure. Really, dangerously, never-ever-use-this insecure.

Consider:

## THIS IS INSECURE; NEVER DO THIS
varname='foo$(touch /tmp/evil)'
foo="Value"
eval "echo \"\$$varname\""

When this is run, it echos Value -- but also creates /tmp/evil. It could instead of run any other attacker-chosen command.

Compare to:

varname='foo$(touch /tmp/evil)'
foo="Value"
echo "${!foo}"

...which does not actually run the command substitution. In pretty much any case where shellshock would have been exploitable (ie. where an attacker could have set an arbitrary environment variable value), using the eval approach would break your security; using the indirection approach is safe(r).

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Thanks for your elaborate explanation and warning, but maybe because it's late, I don't quite get how it relates to my problem. My key confusion is that, in my example, the initial two "echo" return the same thing, so I'm led to believe that the output of both indirection methods is identical. Clearly there's some expansion that I'm not seeing that's affecting how it behaves in the loop case. Could you clarify there? – John de Largentaye May 25 '18 at 02:52
  • Try this: `echo $(echo "first line"; echo "second line")`, and compare it to `echo "$(echo "first line"; echo "second line")"` -- does that make it clear? The lack of quotes is what is causing the command substitution's output to be word-split and passed to `echo` as a series of separate arguments, which `echo` then concatenates with spaces (ignoring the original newlines). – Charles Duffy May 25 '18 at 03:32
  • 1
    ...so, **both indirection methods properly respect the newlines**, but the faulty quoting is then throwing those newlines away. – Charles Duffy May 25 '18 at 03:33