Assumption with double quotes, single quotes & no quotes

Question

In bash :

Using double quotes

echo "*" #Globbing is never done
echo "$variable" # Variable expansion is always done
echo "$(command)" # command expansion is always done

Using single quotes

echo '*' #Globbing is never done 
echo '$variable' # Variable expansion is never done
echo '$(command)' # command expansion is never done

Using no quotes

echo * #Globbing always done.
echo $variable; # Variable expansion is always done
echo $(command) # command expansion is always done

will this apply for all commands?

`'!$"*<>` is typically the order of precedence for those symbols. — 123, Apr 27 '16 at 09:29
Those assumptions are always true -- the places where it gets more interesting is about whether/how other processing steps, such as string-splitting or globbing, are performed on *results* of those expansions. `"$foo"` and `$foo` are, after all, not at all the same thing, despite expansions always occurring in both. — Charles Duffy, May 09 '16 at 21:44
Also, it's important to note that quoting is a per-character attribute, so one can do things like `"$foo"/*` — Charles Duffy, May 09 '16 at 21:45
Can we consider echo $([) as an exception because the command does not really expand correctly as it's incomplete ... but /usr/bin/[ is a command on it own (well on some systems it is on others its a link). — louigi600, May 12 '16 at 12:45
See: [Difference between single and double quotes in bash](http://stackoverflow.com/q/6697753/3776858) — Cyrus, May 12 '16 at 23:55
@Cyrus : Thanks, but the larger interest is knowing whether this apply for all the commands. 123's comment in the beginning is interesting and so it sainthax's answer to this this one. I understand a concrete answer for this one is not possible given it can become very large but I am hopping for a few examples for people who answer this. — sjsam, May 14 '16 at 04:15
@louigi600, no, `echo $([)` is not an exception. That the command in a command expansion *fails* is not at all the same thing as command expansion not being performed. In principle, any command can fail, and some always will fail. Make no mistake: the command `[` failing is a *bona fide* command failure, not a shell syntax error. Additionally, it does not matter for this or most other purposes whether the command that is executed is a shell built-in. — John Bollinger, May 14 '16 at 17:23
So much fun and I got to learn something from other people's answers :-) — SaintHax, May 17 '16 at 10:01

karakfa · Answer 1 · 2016-05-13T11:19:51.840

3

There are multiple forces in place. In general you can assume that single quotes is to hide the contents from bash expansion. Double quotes is to group values which might have white space so that bash sees them as one logical unit but also disable globbing. There are many caveats though...

Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, ', \, and, when history expansion is enabled, !. The characters $ and ' retain their special meaning within double quotes. The backslash retains its special meaning only when followed by one of the following characters: $, ', ", \, or . A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash pre-ceding the ! is not removed.

See section QUOTING from man bash

This example below, will either confuse you further or make it clearer.

$ echo "*test*"
*test*

$ echo '*test*'
*test*

$ msg=$(echo '*test*')
$ echo '$msg'
$msg

$ echo "$msg"
*test*

$ echo $msg
speedtest test1 test2 test3 test4 test5 testdir

note that if there were no matches it would print *test* not empty line as commented by Hastur.

some other interesting tidbits

note that this doesn't work

$ echo 'single quotes don\'t escape'

but this works

$ echo "\"double quotes\" escape"

but you can use one in other without escaping

$ echo '"' "'"

edited May 13 '16 at 11:19

answered May 09 '16 at 21:48

karakfa

66,216
7
41
56

You should also mention that double quotes prevent globbing. It's implicit in the paragraph you quote but your summary misses this half of the common reasons to use quotes. – tripleee May 11 '16 at 07:07
1

__Note:__ `echo $msg` will answer with `*test*` if there will be no matches for the expansion (simple if there are no file with a name compatible it will answer with `*test*` else with `test01 test02 ...` – Hastur May 12 '16 at 13:04
@Hastur This depends on the setting of the `nullglob` shell option: when turned on, an unsuccessful glob match expands to the empty string, when turned off, it expands to itself. – Benjamin W. May 14 '16 at 22:26
You can also end and restart the string to add a single (escaped) quote: `echo 'You can hack single quotes that don'\''t escape in'` – BlakBat May 15 '16 at 10:59
@BenjaminW Right: if `nullglob` is on with no matches, the command will give an empty string, if `nullglob` is off it will return the string `*test*`. __By default it is off.__ Moreover if `failglob` is on (default off) it will be generated an error and print a message. From man bash:`If no matching filenames are found, and the shell option nullglob is not enabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed. If the failglob shell option is set, and no matches are found, an error message is printed and the command is not executed. – Hastur May 15 '16 at 13:04
`echo 'single quotes don\'t escape'` this doesn't work, as explained in the manual though. – SaintHax May 16 '16 at 14:25

SaintHax · Accepted Answer · 2016-05-17T09:58:41.260

It looks like you are looking for exceptions, and I'd guess you have some in mind. I'm going to make the assumption that set -f/set -o noglob are being excluded from this case?

When you use the dd command, globbing will not occure, even if unquoted.

$ ls *.txt
blah.txt  file1.txt  file2.txt  file.txt  logfile.txt
$ dd if=*.txt of=glob.txt
dd: failed to open ‘*.txt’: No such file or directory

Rebuttal and false positives

Here are some examples that are odd, but follow expansion

variable='2010-09-08 12:34:56' echo "$variable" | xargs date +%s -d date: extra operand '12:34:56'

The extra operand shows that variable expansion is happening, you are losing the quotes in the pipe. $ date +%s -d 2010-09-08 12:34:56 date: extra operand ‘12:34:56’

This also happens if you create a script to echo $1 and then expand your quoted variable while passing. It expands, and works as expected. So, the issue is not with xargs, but with your expansion before the pipe which is normal.

Eval... evals whole purpose is to do expansion of its args prior to running a command. Expansion also happens with bash -c, except it takes one argument. So, again, this is not an expansion issue, but a command usage issue.

cmd='printf "%s\n" "$(date -d "$variable" +%c)"' bash -c $cmd

works the same as the expanded version

$ bash -c printf "%s\n" "$(date -d "$variable" +%c)" printf: usage: printf [-v var] format [arguments]

I really enjoyed Hauri's $'...' and $"..." information--however, those are not the samething we are talking about. They are in fact behaving as the bash man page says they should. $'' is as different from '' as (()) is from $(())
I got excited about this one, so... $ ls mclark.txt qt sign_in.txt skel.bash $ zip m*t.zip *t $ ls *.zip m*t.zip

However, this isn't right either-- the splat expands, but upon no match zip uses it as a literal. I found a few commands that did this, but if there was a match (I added a my.zip later) it uses the matched expansion (an error was thrown, b/c my.zip was a text file for testing purposes).

In fact I was ! If time permits, kindly consider adding more. :D — sjsam, May 12 '16 at 13:41
I found a few that glob, but if the glob doesn't match then it uses it as a string literal. I don't think that fits the bill, as it is globbing first, but has a contingency. Very hard question. — SaintHax, May 12 '16 at 22:50
Create files named `if=foo.txt` and `if=bar.txt` and try your `dd` example again. — Barmar, May 14 '16 at 23:29
This answer is long but only gives false information. Downvoted. (show me ONE piece of code that is correctly explained in that answer and I'll consider removing the downvote) — Camusensei, May 20 '16 at 08:42

score 3 · Answer 3 · answered May 16 '16 at 12:19

Short answer: Yes

This asumptions are basicaly true, alway!

variable='2010-09-08 12:34:56' 
vname=variable
date -d "$variable" +%s
1283942096

date -d "${!vname}" +%s
1283942096

date -d $variable +%s
date: extra operand '+%s'
Try 'date --help' for more information.

date -d '$variable' +%s
date: invalid date '$variable'

date -d ${!vname} +%s
date: extra operand '+%s'
Try 'date --help' for more information.

But

Some commands like xargs work precisely about expansion and parameter distribution.

echo "$variable" | xargs date +%s -d
date: extra operand '12:34:56'
Try 'date --help' for more information.

You have to use -0 arg to xargs:

echo "$variable" | xargs -0 date +%s -d
1283942096

Builtin commands could use args differently, especialy eval:

cmd='printf "%s\n" $(date -d "$variable" +%c)'
eval $cmd
Wed
Sep
8
12:34:56
2010

cmd='printf "%s\n" "$(date -d "$variable" +%c)"'
eval "$cmd"
Wed Sep  8 12:34:56 2010

eval $cmd
Wed Sep  8 12:34:56 2010

bash -c "$cmd"
Mon May 16 00:00:00 2016

bash -c $cmd
printf: usage: printf [-v var] format [arguments]

Syntax of funny thing under bash are not limited to "..", '..', and ${}
- $'...' let you print special characters, but don't expand variables:
```
echo $'This\tis\ta string containing ${variable}'
This    is  a string containing ${variable}
```
- Backticks: For compatibility, backtick are always supported. If not very readable, you may see this in some scripts:
```
echo `date +%c -d "${!vname}"`
Wed Sep 8 12:34:56 2010
```
- Syntaxe $"..." could be used for localization:
```
export TEXTDOMAIN=bash
export LANG=fr_CH.utf-8
echo $"Running"
En cours d'exécution
```

eval follows the rules-- it's whole function is to do one layer of expansion prior to a normal execution. It's just as described in the man pages. — SaintHax, May 16 '16 at 23:25

louigi600 · Answer 4 · 2016-05-17T06:20:56.120

2

If nothing matches *.xtx while a.txt is a file mv a.txt *.xtx will get you an unexpected result too. The same applies for other things like cp and even this treats it as quoted:

$ ls *.xtx
/bin/ls: cannot access *.xtx: No such file or directory
$ echo "A" > *.xtx
$ ls *.xtx
*.xtx
$

While they should all return error just like if there were more the one file you would get "ambiguous redirect".

edited May 17 '16 at 06:20

answered May 12 '16 at 14:34

louigi600

716
6
16

`If nothing matches` means globbing is done first. Interesting case though. – sjsam May 14 '16 at 04:19
No destination for a move should just give error and not do anything else. Now unquoted * should allways do globbing so the move should be missing destination and return error ... instead it treats the wildchar as if it was quoted ... so this fits exactly in a command for which the question does not apply. – louigi600 May 16 '16 at 10:05

score 1 · Answer 5 · answered May 10 '16 at 13:19

will this apply for all commands?

Yes.

From Bash reference manuals:

3.1.2.2 Single Quotes

Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

and

3.1.2.3 Double Quotes

Enclosing characters in double quotes (") preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.

The special parameters * and @ have special meaning when in double quotes (see Shell Parameter Expansion).

@louigi600 `$ set -- 1 2 3 4 5 6 7 8 9 0 a b c d e f g h`, `$ echo '${@:7}' "${@:7}"` — Mohit Jain, May 11 '16 at 11:53
~$ echo "*" ; * ; ~$ echo "@" ; @ ; ~$ echo $SHELL ; /bin/bash ; ~$ ; Maybe not always have a special meaning — louigi600, May 11 '16 at 11:54

score 1 · Answer 6 · edited May 11 '16 at 13:04

1

Probably the shell reference manuals and the shell man pages contain the intended behavior ... but the result might not always be what was originally intended.

Reading the "QUOTING" section of the man pages is also intresting. This is a section from the bash man page concerning the single and double quotes: (which is pretty much the same content as the reference manual)

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes. The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or . A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expan- sion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.

edited May 11 '16 at 13:04

Mohit Jain

30,259
8
73
100

answered May 11 '16 at 12:13