Bash Script - iterating over output of find

Question

I have a bash script in which I need to iterate over each line of the ouput of the find command, but it appears that I am iterating over each Word (space delimited) from the find command. My script looks like this so far:

folders=`find -maxdepth 1 -type d`

for $i in $folders
do
    echo $i
done

I would expect this to give output like:

./dir1 and foo
./dir2 and bar
./dir3 and baz

But I am insted getting output like this:

./dir1
and
foo
./dir2
and
bar
./dir3
and
baz

What am I doing wrong here?

See also answer #1 in http://mywiki.wooledge.org/BashPitfalls — Charles Duffy, Feb 09 '14 at 18:44

Charles Duffy · Answer 1 · 2014-02-09T18:42:22.883

24

folders=`foo`

is always wrong, because it assumes that your directories won't contain spaces, newlines (yes, they're valid!), glob characters, etc. One robust approach (which requires the GNU extension -print0) follows:

while IFS='' read -r -d '' filename; do
  : # something with "$filename"
done < <(find . -maxdepth 1 -type d -print0)

Another safe and robust approach is to have find itself directly invoke your desired command:

find . -maxdepth 1 -type d -exec printf '%s\n' '{}' +

See the UsingFind wiki page for a complete treatment of the subject.

edited Feb 09 '14 at 18:42

answered Feb 09 '14 at 18:36

Charles Duffy

280,126
43
390
441

1

Note that the `find ... -exec` form only works for normal commands -- if you want to use shell builtins, shell functions, compound commands, etc you need to use either the `while ... done` approach or the `for` loop chepner suggested. – Gordon Davisson Feb 10 '14 at 01:13
@GordonDavisson Generally true. However, the UsingFind wiki page (linked in my answer) describes using `-exec bash -c '...' _ {} +`, which allows shell builtins and the like to be used. – Charles Duffy Feb 10 '14 at 02:02
Warning! Some commands, including ffmpeg, will read from stdin, and confuse the "read" command used in the loop. Symptoms are beginnings of the filename being cut off (=> invalid names). You need to disable this behaviour in the command that reads from stdin, two solutions here: https://stackoverflow.com/a/21634699 – Seneral May 09 '23 at 18:15

score 5 · Accepted Answer · answered Feb 09 '14 at 18:39

5

Since you aren't using any of the more advanced features of find, you can use a simple pattern to iterate over the subdirectories:

for i in ./*/; do
    echo "$i"
done

answered Feb 09 '14 at 18:39

chepner

497,756
71
530
681

Most answers talk about using while. But, I was trying to find a solution that involved only a for loop and your solution works great. My scenario was to fetch filename from a different folder. So, added a small change to this : `(cd $folder; for i in ./*/; do echo "$i"; done)`. Created a subshell, for running this on my shell without having to change folder paths – Sarfraaz Ahmed Sep 18 '20 at 18:20

Малъ Скрылевъ · Answer 3 · 2014-02-09T19:34:41.353

2

You can do something like this:

find -maxdepth 1 -type d | while read -r i
do
    echo "$i"
done

edited Feb 09 '14 at 19:34

answered Feb 09 '14 at 18:39

Малъ Скрылевъ

16,187
5
56
69

Unsafe. Doesn't correctly handle filenames containing backslash sequences, newlines, runs of multiple whitespace characters, etc. – Charles Duffy Feb 09 '14 at 18:39
@CharlesDuffy fixed and checked both `\`, and whitespaces – Малъ Скрылевъ Feb 09 '14 at 18:45
1

...better -- with the `-r` you've handled backslash sequences, but you're still expanding globs, compressing runs of whitespace into a single character, and stripping trailing whitespace. – Charles Duffy Feb 09 '14 at 18:45
1

@CharlesDuffy how to check it? I've checked two whitespace, it handles that – Малъ Скрылевъ Feb 09 '14 at 18:46
Checked *how*? The problem you still have here is two spaces directly in a row, not two spaces anywhere in the filename. – Charles Duffy Feb 09 '14 at 18:47
@CharlesDuffy do you mean this ' '? or sometihng else? – Малъ Скрылевъ Feb 09 '14 at 18:49
...so, to be clear, the problem is the unquoted expansion passed to the echo. For instance, if `i=$'foo\tbar'`, `echo $i` will print `foo bar`, with a space rather than a tab between the `foo` and the `bar`, because string-splitting breaks it into two separate arguments. – Charles Duffy Feb 09 '14 at 18:51
...but even if you fix that, your answer is _still_ wrong for directory names containing newlines. Yes, they're valid -- try `mkdir -p $'foo\n/etc'` and see what you get. – Charles Duffy Feb 09 '14 at 18:52
@CharlesDuffy `mkdir -p $'foo\n/etc'` this code won't work both in your, and chepner's code. As for the $'foo\tbar': result good in all of the three codes. – Малъ Скрылевъ Feb 09 '14 at 19:09
Incorrect -- what I gave *definitely* works in both my code and chepner's. If you think it doesn't, explain exactly why and how you reach that conclusion. – Charles Duffy Feb 09 '14 at 19:13
@CharlesDuffy I've checked – Малъ Скрылевъ Feb 09 '14 at 19:14
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/47125/discussion-between-charles-duffy-and--) – Charles Duffy Feb 09 '14 at 19:15
Better still. The only bugs this still has -- which both chepner's answer and mine don't -- are unsafe embedded newline handling and stripping of trailing whitespace. – Charles Duffy Feb 09 '14 at 22:02
@CharlesDuffy you second variant too has =) – Малъ Скрылевъ Feb 10 '14 at 05:41
The second variant passes filenames to `printf` with the newline intact -- it's just hard to tell the difference between the printf emitted by newline as part of the filename and the newline emitted by printf according to the format string, so they're still there, so it works perfectly. – Charles Duffy Feb 10 '14 at 12:32

Bash Script - iterating over output of find

3 Answers3

Linked

Related