Quoted parameter expansion with quoted modifier in Bash

Question

I've noticed something weird:

Y=""
echo ${Y:-"\n"}
echo "${Y:-"\n"}"

prints

\n
n

Why is the second line n, not \n? Is this a bug?

It looks as if Bash parsed this as a concatenation of two quoted strings with an unquoted string in between ("${Y:-" and \n and "}") but this doesn't seem to be the case since the commands

echo $(echo "\n")
echo "$(echo "\n")"
echo "${Y:-"'\n'"}"

output

\n
\n
'n'

I'm using GNU bash, version 4.3.11.

you have created an excellent base-line test of `echo`'s interaction with the shell's quoting. I would expect the same results in every bourne-shell derived shell (bash/dash/ash/ksh/zsh and maybe even [t]csh). Everything is working as designed ;-) . Good luck. — shellter, Jun 10 '16 at 16:25
I think the fact that I'm using echo doesn't matter here, does it? As I understand, in these examples Bash evaluates the argument of echo to either the 2-byte string \n or the 1-byte string n or the 3-byte string 'n', and echo doesn't do anything with these strings than just print them. — Jaan, Jun 10 '16 at 16:37
Is there a document that says that "${Y:-"\n"}" should be evaluated to n but ${Y:-"\n"} to \n? It's very counterintuitive, since people usually put quotes around parameter expansions to avoid any transformations by the shell. — Jaan, Jun 10 '16 at 16:41
@shellter Except that isn't what `dash` outputs. In both cases, it outputs a newline, not `\n` or `n`. — chepner, Jun 10 '16 at 18:27
@chepner Because dash's echo interprets backslash sequences by default, unlike bash's echo. Try `printf '%s' string` instead. — MichalH, Jun 10 '16 at 18:37
@Mike I know. That's why I would expect a literal newline in both cases in `dash`, which is what it produces. — chepner, Jun 10 '16 at 18:44

chepner · Answer 1 · 2016-06-10T19:09:37.050

I suspect there is a bug in the handling of the word following :- (in fact, I seem to recall reading something about this, but I can't recall where).

If the value is not quoted, I get results I would expect...

$ echo ${Y:-\n}
n
$ echo "${Y:-\n}"
\n

This is also the result you get in dash (ignoring the fact that dash actually produces a literal newline since POSIX mandates that echo should process escaped characters, something bash only does if you use the non-standard -e option.)

In this example, quoting the default value preserves the backslash. As the result of the parameter expansion produces the backslash, quote removal does not remove it.

$ echo ${Y:-"\n"}   # Equivalent to echo "\n", so the output makes sense
\n

There doesn't seem to be any reason for bash to behave different in this final example just because the entire parameter expansion is being quoted. It is almost as if quote removal is being applied twice, once to remove the outer double quotes and again to incorrectly remove the backslash.

# Quote removal discards the backslash: OK
$ echo \n
n
# Quote removal discards the double quotes: OK
$ echo "n"
n
# Quote removal discards the first backslash after `\\` is recognized
# as a quoted backslash: OK
$ echo \\n
\n 
# Quote removal discards the double quotes, but leaves
# backslash: OK
$ echo "\n"
\n
# Is quote removal discarding both the double quotes *and* the backslash? Not OK
$ echo "${Y:-"\n"}"
n

Related, zsh (with the bsd_echo) option set outputs \n, not n.

% Y=""
% echo "${Y:-"\n"}"
\n

++; I find the behavior with `'\n'` (single quotes) curious as well. — mklement0, Jun 10 '16 at 21:02
Yeah, I think the single quotes should be dropped, but they're not. There are enough comments in `subst.c` around where I think the problem is that I think this is an expected, if never actually diagnosed, problem. — chepner, Jun 11 '16 at 00:24

score 2 · Answer 2 · edited May 23 '17 at 12:23

To complement chepner's helpful answer:

Here's an overview of how the major POSIX-like shells handle the following command:

Y=""
printf '%s\n' ${Y:-"\n"} ${Y:-'\n'} "${Y:-"\n"}" "${Y:-'\n'}"

Note that I've added variations with single quotes.

dash                                    [v0.5.8]
  \n
  \n
  \n
  '\n'

zsh                                     [v5.0.8]
  \n
  \n
  \n
  '\n'

bash                                   [v4.3.42]
  \n
  \n
  n
  '\n'

ksh                                     [93u+]
  \n
  \n
  n
  '\n'

Curiously, in all shells, '\n' inside "..." preserves the single quotes, while removing them in the unquoted case.
With respect to "\n", both bash and ksh exhibit the oddity uncovered by the OP, while dash and zsh do not.

score 0 · Answer 3 · answered Jun 10 '16 at 19:25

0

Maybe I'm looking at this wrong, but I don't seen any inconsistency in the assignment with the default value Y, quoted or unquoted. The echo expression in each case boils down to:

$ echo "\n"
\n

$ echo ""\n""
n

In the first case you have the quoted string "\n", in the second, you have a bare \n (which is simply n)

answered Jun 10 '16 at 19:25

David C. Rankin

81,885
6
58
85

1

Braces start a new context; `"${Y:-"\n"}"` is not three strings concatenated together like `"a"b"c"`. – chepner Jun 10 '16 at 19:32
Then I indeed was looking at it wrong, but what comes out of the new context would appear to be a *quoted* `"\n"`. Otherwise there would appear to be an inconsistency. – David C. Rankin Jun 10 '16 at 19:47
I would add the behavior has been the same since at least `version 3.2.39` to current `4.3.42` – David C. Rankin Jun 10 '16 at 19:56
The same behavior is confirmed in 4.4.0-rc1. The fact that both `zsh` and `dash` behave as if `echo` receives the literal string `\n` in the same situation led me to report this as a bug. – chepner Jun 10 '16 at 19:57
Like you, I remember something about this, whether it was a bug or some strange exception, I'm far too old to recall off the top of my head. I'll follow progress on the bug report. I'm curious what the reason and outcome will be. – David C. Rankin Jun 10 '16 at 20:18

Quoted parameter expansion with quoted modifier in Bash

3 Answers3

Linked