Bash syntax weirdness with files with spaces

Question

Here's a problem you may be familiar with: I wanted to make an inquiry into my disk usage on my Mac, so I tried in my home directory

~/$> for file in `ls`; do du -hs $file; done

with output

 55M    Applications
107M    Desktop
132G    Documents
5.1G    Downloads
 16G    Library
457M    Music
 53M    Pictures
du: VirtualBox: No such file or directory
du: VMs: No such file or directory

which is great except for the last one because it is a directory called "VirtualBox VMs" -- there is a space in the name. I google it (actually, all I had to do was start typing "bash loop..." and it autocompleted to "bash loop over files with spaces" -- thanks Google!) and I get this question, Iterate over list of files with spaces, as the first result.

If you peruse that thread, you'll find a lot of complicated solutions that use xargs or something weird. Of course, I could write my own functions to handle these cases, but I was pretty adamant about having a one line solution. Thus, in my opinion, the hands-down best solution (though not the top-voted or accepted solution) was

for file in *; do du -hs "${file}"; done

Somewhat perversely, the following don't work:

Removing the quotes

for file in *; do du -hs ${file}; done  #splits on spaces and throws error

Using ls, even with the quotes:

for file in `ls`; do du -hs "${file}"; done #splits on spaces and throws error

My question is, Can some bash expert explain to me exactly what the difference between all these is, and why the ones that fail fail?

[BashPitfalls](http://mywiki.wooledge.org/BashPitfalls) would be a place to start. — Charles Duffy, Nov 22 '16 at 19:21
You might also read [ParsingLs](http://mywiki.wooledge.org/ParsingLs). — Charles Duffy, Nov 22 '16 at 19:22
...and reading the explanations of why the faulty code is faulty from https://shellcheck.net/ is another useful place to be. — Charles Duffy, Nov 22 '16 at 19:22
YO shellcheck is a dope tool in addition to being a great resource — Beepboop bebop, Nov 22 '16 at 19:51
@CharlesDuffy: I came on this question to cite exactly those links ... you beat me to it. Put those as an answer, maybe? — Olivier Dulac, Nov 24 '16 at 08:09
@OlivierDulac, ...frankly, I don't think this question *should* be answered -- I'm the vote-to-close (as "too broad", being as it's a conglomeration of individual questions, each of which has a duplicate already in our knowledge base). Feel free to add one yourself. — Charles Duffy, Nov 24 '16 at 17:36

score 1 · Accepted Answer · answered Nov 22 '16 at 19:30

Realize that the working case expands to: du -hs "VirtualBox VMs".

For this case, the issue is that the command expands to:
du -hs VirtualBox VMs
This passes two file names to the du command.
For this case, the issue is that the for command sees two separate tokens. So the iteration iterates over each separately.
du -hs VirtualBox
du -hs VMs

score 0 · Answer 2 · answered Nov 22 '16 at 19:38

0

This is all explained by the fact that word-splitting happens after process substitution and parameter expansion. So the result of $(ls) as well as ${file} undergoes word-splitting, while the double quotes in "${file}" suppress word-splitting.

answered Nov 22 '16 at 19:38

Jens

69,818
15
125
179

Bash syntax weirdness with files with spaces

2 Answers2