Looping through all files in a given directory

Question

Here is what I'm trying to do:

Give a parameter to a shell script that will run a task on all files of jpg, bmp, tif extension.

Eg: ./doProcess /media/repo/user1/dir5

and all jpg, bmp, tif files in that directory will have a certain task run on them.

What I have now is:

for f in *
do
  imagejob "$f"  "output/${f%.output}" ;
done

I need help with the for loop to restrict the file types and also have some way of starting under a specified directory instead of current directory.

kurumi · Answer 1 · 2011-02-28T06:13:54.980

6

Use shell expansion rather than ls

for file in *.{jpg,bmp,tif}
do
  imagejob "$file" "output/${file%.output}"
done

also if you have bash 4.0+, you can use globstar

shopt -s globstar
shopt -s nullglob
shopt -s nocaseglob
for file in **/*.{jpg,bmp,tif}
do
  # do something with $file
done

edited Feb 28 '11 at 06:13

answered Feb 28 '11 at 04:26

kurumi

25,121
5
44
52

+1 You're missing a closing quote. – Dennis Williamson Feb 28 '11 at 05:35
how do i specify the directory to run this script under? – ash Mar 01 '11 at 05:21
im getting an error cannot stat `*.{jpg,bmp,tif}': No such file or directory – ash Mar 02 '11 at 02:04
This solution doesn't work for directories containing a large number of files; it fails with the error "Argument list too long". – Aug 18 '11 at 17:16
I may be mistaken about that, sorry. – Aug 18 '11 at 20:09
For reference, the shell expansion used here is called [Brace Expansion](https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html). It's not defined in POSIX, but is supported by several different shells. Be aware that if no file is found for the pattern, the pattern itself is returned. For example, if there are no files with a `.jpg` extension, the value of `$file`, on the first loop, will be the literal string `*.jpg`. If you have Bash, you can avoid this with the `nullglob` option as shown in the second example. – toxalot Mar 21 '14 at 20:18
@ash The filename expansion is done relative to the current directory. You can specify a subdirectory as part of the pattern. You can also specify an absolute path. For example, `/path/to/files/*.{jpg,bmp,tif}`. – toxalot Mar 21 '14 at 20:20

score 0 · Answer 2 · answered Feb 28 '11 at 04:19

0

for i in `ls $1/*.jpg $1/*.bmp $1/*.tif`; do
    imagejob "$i";
done

This is assuming you're using a bashlike shell where $1 is the first argument given to it.

You could also do:

find "$1" -iname "*.jpg" -or -iname "*.bmp" -or -iname "*.tif" \
          -exec imagejob \{\} \;

answered Feb 28 '11 at 04:19

Borealid

95,191
9
106
122

3

using `ls` to list files with a for loop is bad practice – kurumi Feb 28 '11 at 04:29
The find solution is nice but will find names recursively. The shell globbing method might break with spaces in file names. – Noufal Ibrahim Feb 28 '11 at 04:45
The curly braces don't need to be escaped. – Dennis Williamson Feb 28 '11 at 05:37

score 0 · Answer 3 · answered Feb 28 '11 at 04:21

0

You could use a construct with backticks and ls (or any other commando of course):

for f in `ls *.jpg *.bmp *.tif`; do ...; done

answered Feb 28 '11 at 04:21

markijbema

3,985
20
32

2

using `ls` to list files with a for loop is bad practice – kurumi Feb 28 '11 at 04:29
for what reasons exactly? I know it's a problem with spaces, and very large dirs, but it seems sufficient for other cases. I think your solution is better in this case, but what if I want to use sorting options of ls/pipe its output through another command? – markijbema Feb 28 '11 at 04:42
1

it's to prevent future cases of whitespaces in file names. (Also, its what people call, useless use of `ls`). Why wait for your code to break when you can prevent it in the first place. If you need to use other options coupled with `ls`, use a while read loop (or you can set IFS ) – kurumi Feb 28 '11 at 05:33
See "reasons exactly," see https://mywiki.wooledge.org/ParsingLs – tripleee Nov 26 '18 at 13:46

score 0 · Answer 4 · answered Nov 26 '18 at 13:52

The other solutions here are either Bash-only or recommend the use of ls in spite of it being a common and well-documented antipattern. Here is how to solve this in POSIX sh without ls:

for file in *.jpg *.bmp *.tif; do
    ... stuff with "$file"
done

If you have a very large number of files, perhaps you also want to look into

find . -maxdepth -type f \( \
    -name '*.jpg' -o -name '*.bmp' -o -name '*.tif' \) \
    -exec stuff with {} +

which avoids the overhead of sorting the file names alphabetically. The -maxdepth 1 says to not recurse into subdirectories; obviously, take it out or modify it if you do want to recurse into subdirectories.

The -exec ... + predicate of find is a relatively new introduction; if your find is too old, you might want to use -exec ... \; or replace the -exec stuff with {} + with

find ... -print0 |
xargs -r0 stuff with

where however again the -print0 option and the corresponding -0 option for xargs are a GNU extension.

Looping through all files in a given directory

4 Answers4