Looping over directories in Bash

Question

I have a fundamental question about how bash works, and a related practical question.

Fundamental question: suppose I am in a directory that has three subdirectories: a, b, and c.

hen the code

for dir in $(ls)
do 
    echo $dir
done

spits out:

a b c
a b c
a b c

i.e, dir always stores a list of all of the files/directories in my cwd. My question is: why in the world would this be convenient? In my opinion it is far more useful and intuitive to have dir store each element at a time, i.e I would want to have output

a
b
c

Also, as per one of the answers - it is wrong to use for dir in $(ls), but when I use for dir in $(ls -l) I get even more copies of a b c (more than there are directories/files in the cwd). Why is that?

My second question is practical: how do I loop over all the directories (not files!) in my cwd that start with capital W? I started with

for dir in `ls -l W*`

but this fails because a) the reason in question 1 and b) because it doesn't exclude files. Suggestions appreciated.

Your code snippet doesn't give the result you say it does. Please reproduce the actual code more carefully. — Gordon Davisson, Apr 06 '13 at 18:42
You are right. When I was first trying this code I was using `$(ls)`, but when I wrote up the question I changed `$(ls)` to `$(ls -l)` because that seems to be how it's done everywhere I've looked. I assumed that the output would be the same and am surprised that it's not. — alexvas, Apr 06 '13 at 18:51
@alexvas Why are you surprised that the output of `ls -l` is different compared to just `ls`? The parameter changes its behaviour (type `man ls` for details): `-l` gives you long output, including permissions, ownership, date etc. This is an even bigger no-no to parse, because its output can differ immensely depending on the implementation and your `locale`. `ls -l` is what you would use on the command line, but never in a script like this. If you have seen code that parses its output in your environment you should contact the author and ask for it to be fixed. — Adrian Frühwirth, Apr 08 '13 at 07:45
Right, I mean that I am surprised that the format of the output is _so_ different. The output of the `$(ls)` case yields three rows of `a b c`, while the output of `$(ls -l)` yields three rows each containing the desired output (`a`, followed by `b`, followed by `c`). I figured the commands are ... similar? ... so should produce reasonably ... similar? ... output. — alexvas, Apr 08 '13 at 20:24
Possible duplicate of http://stackoverflow.com/questions/2107945/how-to-loop-over-directories-in-linux — tripleee, Oct 19 '16 at 11:48
@tipleee One part of my question is identical to the one you linked, but I asked two additional parts: a conceptual question about the output of ls and I also added the requirement that directories start with a specific character. — alexvas, Oct 19 '16 at 23:16

Adrian Frühwirth · Accepted Answer · 2013-04-06T19:21:38.250

44

Never ever parse the output of ls like this (Why you shouldn't parse the output of ls(1)).

Also, your syntax is wrong. You don't mean (), you mean $().

That being said, to loop over directories starting with W you would do (or use the find command instead, depending on your scenario):

for path in /my/path/W*; do
    [ -d "${path}" ] || continue # if not a directory, skip
    dirname="$(basename "${path}")"
    do_stuff
done

As for the output you get from the evil ls-loop, it should not look like that. This is the expected output and demonstrates why you do not want to use ls in the first place:

$ find
.
./c
./a
./foo bar
./b

$ type ls
ls is hashed (/bin/ls)

$ for x in $(ls); do echo "${x}"; done
a
b
c
foo
bar

edited Apr 06 '13 at 19:21

answered Apr 06 '13 at 18:40

Adrian Frühwirth

42,970
10
60
71

Thanks. I want to know, though, where my understanding of `for dir in $(ls)` is wrong. I assumed that `ls` returns an array storing the directories and files in my cwd, and then `for` would loop over them. Why is this not happening? – alexvas Apr 06 '13 at 18:57
It does not return an array, it prints out a string to stdout which you capture with the `$()` syntax. Other than that I cannot answer your question because your for loop should not print a matrix like you posted, based on the scenario you mentioned. – Adrian Frühwirth Apr 06 '13 at 19:17
Updated my answer to show that the output should actually look like you expected in the first place, you might want to try this again. – Adrian Frühwirth Apr 06 '13 at 19:23
Ah! But there is a difference between how you and I echo the names of the directories! I used `echo $dir` and you used `echo ${dir}`. Indeed, if I use your syntax then I get similar output to yours, and I am certain that if you use my syntax then you will get my output. What do the curly braces mean, then? – alexvas Apr 06 '13 at 19:42
No, they are equivalent in this case. The curly braces are needed in scenarios like this: `foo=hello; echo "$fooworld"; echo "${foo}world"` (in cases variables name are ambiguous). I suggest you read the [Advanced Bash Scripting Guide](http://tldp.org/LDP/abs/html/) and the [BashGuide](http://mywiki.wooledge.org/BashGuide) to learn more about the basics, common pitfalls and basically everything you need/want to know about bash scripting. – Adrian Frühwirth Apr 06 '13 at 19:50
@alexvas: I tried with `$dir`, but I still don't get the output you describe. Are you sure you're using *exactly* the code you give in the question? – Gordon Davisson Apr 06 '13 at 20:16
@AdrianFrühwirth sorry aboout that, may be my eye was feeling dizzy, deleted comment.... – Jahid Jul 09 '15 at 12:54

score 12 · Answer 2 · edited May 08 '18 at 20:07

This should work:

shopt -s nullglob   # empty directory will return empty list
for dir in ./*/;do
    echo "$dir"         # dir is directory only because of the / after *
done

To be recursive in subdirectories too, use globstar:

shopt -s globstar nullglob
for dir in ./**/;do
    echo "$dir" # dir is directory only because of the / after **
done

You can make @Adrian Frühwirths' method to be recursive to sub-directories by using globstar too:

shopt -s globstar
for dir in ./**;do
    [[ ! -d $dir ]] && continue # if not directory then skip
    echo "$dir"
done

From Bash Manual:

globstar

If set, the pattern ‘**’ used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a ‘/’, only directories and subdirectories match.

nullglob

If set, Bash allows filename patterns which match no files to expand to a null string, rather than themselves.

score 0 · Answer 3 · edited Jul 09 '15 at 02:21

0

Well, you know what you are seeing is not what you are expecting. The output you are seeing is not from the echo command, but from the dir command.

Try the following:

ls -1 | while read line; do 

   if [-d "$line" ] ; then 
      echo $line
   fi

done


for files in $(ls) ; do

   if [-d "$files" ] ; then 
      echo $files
   fi

done

edited Jul 09 '15 at 02:21

Pang

9,564
146
81
122

answered Apr 06 '13 at 20:38

user1587276

906
6
4

Looping over directories in Bash

3 Answers3

Linked

Related