Bash eval replacement $() not always equivalent?

Question

Everybody says eval is evil, and you should use $() as a replacement. But I've run into a situation where the unquoting isn't handled the same inside $().

Background is that I've been burned too often by file paths with spaces in them, and so like to quote all such paths. More paranoia about wanting to know where all my executables are coming from. Even more paranoid, not trusting myself, and so like being able to display the created commands I'm about to run.

Below I try variations on using eval vs. $(), and whether the command name is quoted (cuz it could contain spaces)

  BIN_LS="/bin/ls"
  thefile="arf"
  thecmd="\"${BIN_LS}\" -ld -- \"${thefile}\""

  echo -e "\n    Running command   '${thecmd}'"
  $($thecmd)

          Running command   '"/bin/ls" -ld -- "arf"'
      ./foo.sh: line 8: "/bin/ls": No such file or directory

  echo -e "\n    Eval'ing command  '${thecmd}'"
  eval $thecmd

          Eval'ing command  '"/bin/ls" -ld -- "arf"'
      /bin/ls: cannot access arf: No such file or directory

  thecmd="${BIN_LS} -ld -- \"${thefile}\""

  echo -e "\n    Running command   '${thecmd}'"
  $($thecmd)

          Running command   '/bin/ls -ld -- "arf"'
      /bin/ls: cannot access "arf": No such file or directory

  echo -e "\n    Eval'ing command  '${thecmd}'"
  eval $thecmd

          Eval'ing command  '/bin/ls -ld -- "arf"'
      /bin/ls: cannot access arf: No such file or directory

  $("/bin/ls" -ld -- "${thefile}")

      /bin/ls: cannot access arf: No such file or directory

So... this is confusing. A quoted command path is valid everywhere except inside a $() construct? A shorter, more direct example:

$ c="\"/bin/ls\" arf"
$ $($c)
-bash: "/bin/ls": No such file or directory
$ eval $c
/bin/ls: cannot access arf: No such file or directory
$ $("/bin/ls" arf)
/bin/ls: cannot access arf: No such file or directory
$ "/bin/ls" arf
/bin/ls: cannot access arf: No such file or directory

How does one explain the simple $($c) case?

score 17 · Answer 1 · answered Oct 08 '12 at 20:49

17

The use of " to quote words is part of your interaction with Bash. When you type

$ "/bin/ls" arf

at the prompt, or in a script, you're telling Bash that the command consists of the words /bin/ls and arf, and the double-quotes are really emphasizing that /bin/ls is a single word.

When you type

$ eval '"/bin/ls" arf'

you're telling Bash that the command consists of the words eval and "/bin/ls" arf. Since the purpose of eval is to pretend that its argument is an actual human-input command, this is equivalent to running

$ "/bin/ls" arf

and the " gets processed just like at the prompt.

Note that this pretense is specific to eval; Bash doesn't usually go out of its way to pretend that something was an actual human-typed command.

When you type

$ c='"/bin/ls" arf'
$ $c

the $c gets substituted, and then undergoes word splitting (see §3.5.7 "Word Splitting" in the Bash Reference Manual), so the words of the command are "/bin/ls" (note the double-quotes!) and arf. Needless to say, this doesn't work. (It's also not very safe, since in addition to word-splitting, $c also undergoes filename-expansion and whatnot. Generally your parameter-expansions should always be in double-quotes, and if they can't be, then you should rewrite your code so they can be. Unquoted parameter-expansions are asking for trouble.)

When you type

$ c='"/bin/ls" arf'
$ $($c)

this is the same as before, except that now you're also trying to use the output of the nonworking command as a new command. Needless to say, that doesn't cause the nonworking command to suddenly work.

As Ignacio Vazquez-Abrams says in his answer, the right solution is to use an array, and handle the quoting properly:

$ c=("/bin/ls" arf)
$ "${c[@]}"

which sets c to an array with two elements, /bin/ls and arf, and uses those two elements as the word of a command.

answered Oct 08 '12 at 20:49

ruakh

175,680
26
273
307

(hate the 5 min limit) That last line is not showing execution of a packaged command in a form where output could be captured (which though not said in original question is needed) Try c=("/bin/ls" "/bin/l*") and c=("/bin/ls" /bin/l*) with echo "'${c[@]}'". This method won't let me build up an array with quoted arguments for later use with $() ... ? – Shenme Oct 08 '12 at 21:16
1

@Shenme: As far as I can tell, what you're saying is, "I want to use `eval`, because I want to process an arbitrary string as a Bash command." What everyone else is saying is, "`eval` is evil, because it processes an arbitrary string as a Bash command." You're trying to reconcile these viewpoints by asking, "Is there some *non*-evil way to do this evil thing?", but that won't work. The viewpoints are fundamentally irreconcilable. To do an evil thing, use an evil tool; or, to avoid the evil tool, rewrite your script to avoid doing the evil thing. – ruakh Oct 08 '12 at 21:32
No, no arbitrary commands, no evil intentions. :-) Just wanted to build up a command and execute it in a controlled manner. I will control the inputs, and am even paranoid enough to use '--'. People see the simplistic answers to complex meta-questions, and criticize you if you 'do' use eval, so I wanted to avoid the criticism. :-( Anyway, z=$(${c[@]}) works just fine. – Shenme Oct 08 '12 at 21:35
1

@Shenme: FWIW, I'm not really criticizing you for using `eval`. I've used it before, and I've also used its equally-evil analogues in Perl and JavaScript. The point of identifying something as evil is not so that we can burn its practitioners at the stake, but so that we can recognize that it's a bad sign -- a sign that something *could* be done better, if we were willing to put in the effort to do so -- and an omen that evil things will issue, because it's difficult or impossible to envision all possible results. – ruakh Oct 08 '12 at 21:50
I'm dumbfounded as to how this answer has not been marked as correct yet. Kudos to you, @ruakh. You just saved me a lot of Bash debugging. – SimonC Feb 06 '19 at 15:23
1

in your answer, could you elaborate a little bit on the case where someone would try to simply remove the double quotes, like this: `c='/bin/ls arf'` then `$($c)` ? Since it also _seemingly_ gives the correct output. – Magne May 04 '21 at 12:46
>> Needless to say, this doesn't work. I actually just want to know why it doesn't work? Why double quote doesn't work. wordspliting separate the command("/bin/ls") and args(test). It should work as expected cause `"/bin/ls" test` works fine in terminal – Stan Mar 20 '22 at 00:00
@StanPeng: This answer was my best attempt to answer exactly that question. If you've read this answer, and it didn't help you understand, then I don't have anything better up my sleeve for you. :-/ – ruakh Mar 20 '22 at 00:07
I just guess quoted command only works in interactive terminal – Stan Mar 20 '22 at 00:29

score 5 · Answer 2 · answered Oct 08 '12 at 20:47

5

With the fact that it doesn't make sense in the first place. Use an array instead.

$ c=("/bin/ls" arf)
$ "${c[@]}"
/bin/ls: cannot access arf: No such file or directory

answered Oct 08 '12 at 20:47

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

Unquoted strings get evaluated immediately. c=("/bin/ls" /bin/l*) does something quite different than wanted. (look with echo "'${c[@]}'" ) c=("/bin/ls" "/bin/l*") then z=$("${c[@]}") fails. An array of arguments can't be used nicely with $() ? – Shenme Oct 08 '12 at 21:24
1

However, c=("/bin/ls" "/bin/l*") then z=$(${c[@]}) does seem to work just fine. Is this the executable form $() you meant to show me? – Shenme Oct 08 '12 at 21:33

bta · Answer 3 · 2012-10-08T21:12:07.167

From the man page for bash, regarding eval:

eval [arg ...]: The args are read and concatenated together into a single command. This command is then read and executed by the shell, and its exit status is returned as the value of eval.

When c is defined as "\"/bin/ls\" arf", the outer quotes will cause the entire thing to be processed as the first argument to eval, which is expected to be a command or program. You need to pass your eval arguments in such a way that the target command and its arguments are listed separately.

The $(...) construct behaves differently than eval because it is not a command that takes arguments. It can process the entire command at once instead of processing arguments one at a time.

A note on your original premise: The main reason that people say that eval is evil was because it is commonly used by scripts to execute a user-provided string as a shell command. While handy at times, this is a major security problem (there's typically no practical way to safety-check the string before executing it). The security problem doesn't apply if you are using eval on hard-coded strings inside your script, as you are doing. However, it's typically easier and cleaner to use $(...) or `...` inside of scripts for command substitution, leaving no real use case left for eval.

Additional tip: when trying to see how an expanded command is seen by the shell, using `set -v` can sometimes give more accurate results than `echo`. — bta, Oct 08 '12 at 21:16
so basically, `eval` is safe if the evaluated string is not exposed in any way to user input; but if we fail at any point on preventing that, it can be exploited; therefore using ``...`` or `$(...)` requires less precautions compared to `eval`; thx man! this was the specific tip I was looking for before changing my scripts hehe! :) — Aquarius Power, Jul 02 '14 at 01:54

Stan · Answer 4 · 2022-03-20T00:58:51.707

Using set -vx helps us understand how bash process the command string. As seen in the picture, "command" works cause quotes will be stripped when processing. However, when $c(quoted twice) is used, only the outside single quotes are removed. eval can process the string as the argument and outside quotes are removed step by step. It is probably just related to how bash semanticallly process the string and quotes.