Why does \$ reduce to $ inside backquotes [though not inside $(...)]?

Question

Going over the POSIX standard, I came across another rather technical/pointless question. It states:

Within the backquoted style of command substitution, <backslash> shall retain its literal meaning, except when followed by: '$' , '`' , or <backslash>.

It's easy to see why '`' and '\' lose their literal meanings: nested command substitution demands a "different" backquote inside the command substitution, which in turn forces '\' to lose its literal meaning. So, for instance, the following different behavior seems reasonable:

$ echo $(echo \\\\)
\\
$ echo `echo \\\\`
\

But what about '$'? I.e., what's the point or, more concretely, a possible benefit of the following difference?

$ echo $(echo \$\$)
$$
$ echo `echo \$\$`
4735

As '$' by itself is not ruled out inside backquotes, it looks like you would use either '$' or '\\\$' all the time, but never the middle case '\$'.

To recap,

$ echo `echo $$` # PID, OK
4735
$ echo `echo \\\$\\\$` # literal "$$", OK
$$
$ echo `echo \$\$` # What's the point?
4735

PS: I know this question is rather technical... I myself go for the more modern $(...) substitution all the time, but I'm still curious.

Answer 1

By adding a \, you make the inner subshell expand it instead of the outer shell. A good example would be to actually force the starting of a new shell, like this:

$ echo $$
4988
$ echo `sh -c 'echo $$'`
4988
$ echo `sh -c 'echo \$\$'`
4990
$ echo `sh -c 'echo \\\$\\\$'`
$$

Answer 2

Basic Answer

Consider the following command, which finds the base directory where gcc was installed:

gcc_base=$(dirname $(dirname $(which gcc)))

With the $(...) notation, there is no problem with the parsing; it is trivial and is one of the primary reason why the notation is recommended. The equivalent command using back-ticks is:

gcc_base=`dirname \`dirname \\\`which gcc\\\`\``

When the shell first parses this command, it encounters the first backtick, and has to find the matching close backtick. That's when the quoted section comes into effect:

Within the backquoted style of command substitution, shall retain its literal meaning, except when followed by: '$' , '`' , or .

gcc_base=`dirname \`dirname \\\`which gcc\\\`\``
                  ^         ^ ^          ^ ^ ^
                  1         2 3          4 5 6

1. backslash-backtick - special rule
2. backslash-backslash - special rule
3. backslash-backtick - special rule
4. backslash-backslash - special rule
5. backslash-backtick - special rule
6. backslash-backtick - special rule

So, the unescaped backtick at the end marks the end of the outermost backtick command. The sub-shell that processes that command sees:

dirname `dirname \`which gcc\``

The backslash-back escapes are given the special treatment again, and the sub-sub-shell sees:

dirname `which gcc`

• The sub-sub-sub-shell gets to see which gcc and evaluates it (e.g. /usr/gcc/v4.6.1/bin/gcc).
• The sub-sub-shell evaluates dirname /usr/gcc/v4.6.1/bin/gcc and produces /usr/gcc/v4.6.1/bin.
• The sub-shell evaluates dirname /usr/gcc/v4.6.1/bin and produces /usr/gcc/v4.6.1.
• The shell assigns /usr/gcc/v4.6.1 to gcc_base.

In this example, the backslashes were only followed by the special characters - backslash, backtick, dollar. A more complex example would have, for example, \" sequences in the command, and then the special rule would not apply; the \" would simply be copied through unchanged and passed to the relevant sub-shell(s).

Extraordinarily Complex Stuff

For example, suppose you had a command with a blank in its name (heaven forbid; and this shows why!) such as totally amazing (two blanks; it is a more stringent test than a single blank). Then you could write:

$ cmd="totally  amazing"
$ echo "$cmd"
totally  amazing
$ which "$cmd"
/Users/jleffler/bin/totally  amazing
$ dirname $(which "$cmd")
usage: dirname path
$ # Oops!
$ dirname "$(which \"\$cmd\")"
"$cmd": not found
.
$ # Oops!
$ dirname "$(which \"$cmd\")"
"totally: not found
amazing": not found
.
$ dirname "$(eval which \"$cmd\")"
totally amazing: not found
.
$ dirname "$(eval which \"\$cmd\")"
/Users/jleffler/bin
$ # Ouch, but at least that worked!
$ # But how to extend that to the next level?
$ dirname "$(eval dirname \"\$\(eval which \\\"\\\$cmd\\\"\)\")"
/Users/jleffler
$

OK - well, that's the "easy" one! Do you need a better reason to avoid spaces in command names or path names? I've also demonstrated to my own satisfaction that it works correctly with pathnames that contain spaces.

So, can we compress the learning cycle for backticks? Yes...

$ cat x3.sh
cmd="totally  amazing"
which "$cmd"
dirname "`which \"$cmd\"`"
dirname "`dirname \"\`which \\"\$cmd\\\"\`\"`"
$ sh -x x3.sh
+ cmd='totally  amazing'
+ which 'totally  amazing'
/Users/jleffler/bin/totally  amazing
++ which 'totally  amazing'
+ dirname '/Users/jleffler/bin/totally  amazing'
/Users/jleffler/bin
+++ which 'totally  amazing'
++ dirname '/Users/jleffler/bin/totally  amazing'
+ dirname /Users/jleffler/bin
/Users/jleffler
$

That is still a ghastly, daunting, non-intuitive set of escape sequences. It's actually shorter than the version for $(...) notation, and doesn't use any eval commands (which always complicate things).

Answer 3

This probably has to do with the strange way the Bourne shell parses substitutions (the real Korn shell is slightly similar but most other shells do not exhibit the strange behaviour at all).

Basically, the Bourne shell's parser does not interpret substitutions ($ and `) inside double-quotes, or parameter substitution ($) anywhere. This is only done at expansion time. Also, in many cases unmatched quotes (single-quotes, double-quotes or backquotes) are not an error; the closing quote is assumed at the end.

One consequence is that if a parameter substitution with a word containing spaces like ${v+a b} occurs outside double-quotes, it is not parsed correctly and will cause an expansion error when executed. The space needs to be quoted. Other shells do not have this problem.

Another consequence is that double-quotes inside backquotes inside double-quotes do not work reliably. For example,

v=0; echo "`v=1; echo " $v "`echo b"

will print

 1 echo b

in most shells (one command substitution), but

 0 b

in the Bourne shell and the real Korn shell (ksh93) (two command substitutions).

(Ways to avoid the above issue are to assign the substitution to a variable first, so double-quotes are not necessary, or to use new-style command substitution.)

The real Korn shell (ksh93) attempts to preserve much of the strange Bourne shell behaviour but does parse substitutions at parse time. Thus, ${v+a b} is accepted but the above example has "strange" behaviour. A further strange thing is that something like

echo "`${v+pwd"

is accepted (the result is like with the missing closing brace). And where does the opening brace in the error message from

echo "`${v+pwd`"

come from?

The below session shows an obscure case where $ and \$ differ in a non-obvious way:

$ echo ${.sh.version}
Version JM 93u 2011-02-08
$ v=0; echo "`v=1; echo "${v+p q}"`echo b" 
p qecho b
$ v=0; echo "`v=1; echo "\${v+p q}"`echo b" 
p{ q}b

Answer 4

Basically, a backslash is an escape character. You put it before another character to represent something special. An 'n','t','$' and '\'are these special characters.

"\n" --> newline
"\t" --> tab (indent)
"\$" --> $ (because a $ before a word in shell denotes a variable)
"\\" --> \ 

The backslash before characters is only interpreted the above way when it is inside quotes.

If you want to find more info or other escape chars go here