Exactly why does result=$($shell_command) fail?

Question

Why does assigning command output work in some cases and seemingly not in others? I created a minimal script to show what I mean, and I run it in a directory with one other file in it, a.txt. Please see the ??? in the script below and let me know what's wrong, perhaps try it. Thanks.

#!/bin/bash

## setup so anyone can copy/paste/run this script ("complete" part of MCVE)
tempdir=$(mktemp -d "${TMPDIR:-/tmp}"/demo.XXXX) || exit # make a temporary directory
trap 'rm -rf "$tempdir"' 0 # delete temporary directory on exit
cd "$tempdir" || exit      # don't risk changing non-temporary directories
touch a.txt                # create a sample file

cmd1="find . -name 'a*' -print"
eval $cmd1         # this produces "./a.txt" as expected
res1=$($cmd1)
echo "res1=$res1"  # ??? THIS PRODUCES ONLY "res1=" , $res1 is blank  ???

# let's try this as a comparison

cmd2="ls a*"
res2=$($cmd2)
echo "res2=$res2"  # this produces "res2=a.txt"

This is not the way to go, see [I'm trying to put a command in a variable, but the complex cases always fail!](http://mywiki.wooledge.org/BashFAQ/050). In your [previous question](http://stackoverflow.com/q/31755573/1983854) you were looking for something and it wasn't very clear what. Now it looks less clear. Maybe try to explain your ultimate goal so that you don't enter in strange paths. — fedorqui, Jul 31 '15 at 23:37
The last question is good, and done. This is a new question, with the simplified script very close to what I want to do somewhere else in my code. — Ray, Jul 31 '15 at 23:39
Thanks for your reference, but I'm not trying to do something hugely complex here. Perhaps you can run the above script and tell me what's wrong. — Ray, Jul 31 '15 at 23:40
The current approach is quite bad, I would recommend checking it against http://www.shellcheck.net/ to find basic recommendations. And this question is like "I am doing , how can I fix it?" better say "I want to do and here is how I am approaching it" — fedorqui, Jul 31 '15 at 23:42

Charles Duffy · Accepted Answer · 2015-08-01T02:32:07.597

Let's look at exactly what this does:

cmd1="find . -name 'a*' -print"
res1=$($cmd1)
echo "res1=$res1"  # ??? THIS PRODUCES ONLY "res1=" , $res1 is blank  ???

As per BashFAQ #50, execution of res1=$($cmd1) does the following, assuming you have no files with names starting with 'a and ending with ' (yes, with single quotes as part of the name), and that you haven't enabled the nullglob shell option:

res1=$( find . -name "'a*'" -print )

Note the quoting around than name? That quoting represents that the 's are treated as data, rather than syntax; thus, rather than having any effect on whether the * is expanded, they're simply an additional element required to be part any filename for it to match, which is why you get a result with no matches at all. Instead, as the FAQ tells you, use a function:

cmd1() {
  find . -name 'a*' -print
}
res1=$(cmd1)

...or an array:

cmd1=( find . -name 'a*' -print )
res1=$( "${cmd1[@]}" )

Now, why does this happen? Read the FAQ for a full explanation. In short: Parameter expansion happens after syntactic quotes have already been applied. This is actually a Very Good Thing from a security perspective -- if all expansions recursively ran through full parsing, it would be impossible to write secure code in bash handling hostile data.

Now, if you don't care about security, and you also don't care about best practices, and you also don't care about being able to correctly interpret results with unusual filenames:

cmd1="find . -name 'a*' -print"
res1=$(eval "$cmd1") # Force parsing process to restart from beginning. DANGEROUS if cmd1
                     # is not static (ie. constructed with user input or filenames);
                     # prone to being used for shell injection attacks.
echo "res1=$res1"

...but don't do that. (One can get away with sloppy practices only until one can't, and the point when one can't can be unpleasant; for the sysadmin staff at one of my former jobs, that point came when a backup-maintenance script deleted several TB worth of billing data because a buffer overflow had placed random garbage in the name of a file that was due to be deleted). Read the FAQ, follow the practices it contains.

Exactly why does result=$($shell_command) fail?

1 Answers1