187
for i in $(ls);do
    if [ $i = '*.java' ];then
        echo "I do something with the file $i"
    fi
done

I want to loop through each file in the current folder and check if it matches a specific extension. The code above doesn't work, do you know why?

codeforester
  • 39,467
  • 16
  • 112
  • 140
AR89
  • 3,548
  • 7
  • 31
  • 46

7 Answers7

294

No fancy tricks needed:

for i in *.java; do
    [ -f "$i" ] || break
    ...
done

The guard ensures that if there are no matching files, the loop will exit without trying to process a non-existent file name *.java.

In bash (or shells supporting something similar), you can use the nullglob option to simply ignore a failed match and not enter the body of the loop.

shopt -s nullglob
for i in *.java; do
    ...
done

Some more detail on the break-vs-continue discussion in the comments. I consider it somewhat out of scope whether you use break or continue, because what the first loop is trying to do is distinguish between two cases:

  1. *.java had no matches, and so is treated as literal text.
  2. *.java had at least one match, and that match might have included an entry named *.java.

In case #1, break is fine, because there are no other values of $i forthcoming, and break and continue would be equivalent (though I find break more explicit; you're exiting the loop, not just waiting for the loop to exit passively).

In case #2, you still have to do whatever filtering is necessary on any possible matches. As such, the choice of break or continue is less relevant than which test (-f, -d, -e, etc) you apply to $i, which IMO is the wrong way to determine if you entered the loop "incorrectly" in the first place.

That is, I don't want to be in the position of examining the value of $i at all in case #1, and in case #2 what you do with the value has more to do with your business logic for each file, rather than the logic of selecting files to process in the first place. I would prefer to leave that logic to the individual user, rather than express one choice or the other in the question.


As an aside, zsh provides a way to do this kind of filtering in the glob itself. You can match only regular files ending with .java (and disable the default behavior of treating unmatched patterns as an error, rather than as literal text) with

for f in *.java(.N); do
  ...
done

With the above, you are guaranteed that if you reach the body of the loop, then $f expands to the name of a regular file. The . makes *.java match only regular files, and the N causes a failed match to expand to nothing instead of producing an error.

There are also other such glob qualifiers for doing all sorts of filtering on filename expansions. (I like to joke that zsh's glob expansion replaces the need to use find at all.)

chepner
  • 497,756
  • 71
  • 530
  • 681
  • How to change this to be able to loop over all *.java and *.cpp files? – Temak May 13 '15 at 13:06
  • 6
    The simplest way is to add another pattern: `for i in *.java *.cpp; do`. If you have extended patterns enabled in `bash` with `shopt -s extglob`, you can write `for i in *.@(java|cpp); do`. – chepner May 13 '15 at 13:37
  • 8
    It will if it actually matches any files. You need to use `shopt -s nullglob` so that a non-matching pattern expands to the empty sequence rather than be treated literally. – chepner Aug 01 '15 at 23:08
  • Does this loop through *.java files in current directory? – zygimantus Jan 05 '16 at 10:32
  • 1
    @zygimantus yes it should, so long as current directory is where the script is running from. If you are not in the directory you want to be though, you should `cd` to that directory before you start the `for` loop – danielsdesk Oct 27 '16 at 16:58
  • I think this fails if there is no *.java files present, then it treats it as a string – puk Jul 07 '17 at 17:11
  • 1
    @puk It depends on your shell options. By default, a non-matching pattern is treated as a literal string. You can set `nullglob` to have it treated as an empty sequence, or set `failglob` to have it treated as an error. (If you set both, `failglob` takes precedence.) – chepner Jul 07 '17 at 17:31
  • 3
    @chepner It might be useful for non experts to have this indicated in the answer as someone may copy paste it and find it not working. A perfect example would be someone who is converting all "*.jpg" files to "*.png" before carrying out a crucial function – puk Jul 07 '17 at 19:50
  • 5
    Instead of `[ -f "$i" ] || break`, we need `[ -f "$i" ] || continue`, right? – codeforester May 05 '18 at 16:36
  • I guess `break` and `continue` would have the same effect with just one pattern which doesn't match. With multiple patterns (`for i in *.java *.foo *.bar`, e.g.), you would want `continue` to move on to the next pattern. – chepner May 05 '18 at 21:10
  • 4
    @codeforester is right, `continue` is necessary. If there is a plain file named `b.java` and a directory named `a.java`, then the loop can be terminated by `break` before it reaches `b.java`. – nekketsuuu Jun 21 '18 at 07:56
  • This `nullglob` default setting is the most counter intuitive thing in bash I have seen so far. And it contains a lot of non intuitive things... – kap May 11 '20 at 15:19
  • @kap Like so many other things, it's probably the result of having to support historical decisions that assumed (I'm guessing) a user wouldn't use a glob if they didn't expect it to produce at least one match. The thinking (again, I'm guessing) was that it was simpler to let whatever command was going use the output to produce an error than trying to deal with failures to match themselves. – chepner May 11 '20 at 15:47
  • @chepner May I ask why the editing of `break` to `continue` was rejected as it "deviates from the original intent of the post"? By changing the keyword to `continue` the guard will still `ensure that (...) the loop will exit without trying to process a non-existent file name *.java`. Right now, if there is a directory named `a.java`, the file `b.java` will not be considered by the loop, but with `continue` it _will_ be considered. Therefore I would consider the edit to have the same intent, but it improves the loop imo – Aron Hoogeveen May 20 '22 at 14:56
  • I might consider switch to using `-e` to allow any kind of file-system entry to be processed, but I would still prefer to use `break`. The point of the answer, though, was to show that you could use a pattern to match the desired files, rather than iterating over *everything* in the current directory. I don't really see the exact details of the filtering being in the scope of the answer. – chepner May 20 '22 at 15:03
  • The original poster stated "I want to loop through **each** file in the current folder", so not iterating over _everything_ in the current folder does not answer the question then. Others having his/her problem and reading this answer might think it actually considers _all_ files in the folder, when it in fact might not consider all files. (BTW I do really appreciate both your post and your quick answer :)) – Aron Hoogeveen May 20 '22 at 15:10
  • 1
    Really, the issue is not so much over `break` or `continue`, but in whether `*.java` represents the failure of a pattern to expand or an *actual* entry named `*.java`. If it's not a real entry, then you want to break; otherwise, who knows what you want to do. If I'm going to edit the answer, I'd like something that addresses the success-vs-failure of the expansion, not what you do with `$i`. (That's why I really prefer the `nullglob` answer; it lets you not enter the loop at all rather than specify how to terminate the loop early when necessary.) – chepner May 20 '22 at 15:13
30

Recursively add subfolders,

for i in `find . -name "*.java" -type f`; do
    echo "$i"
done
luismartingil
  • 1,029
  • 11
  • 16
23

Loop through all files ending with: .img, .bin, .txt suffix, and print the file name:

for i in *.img *.bin *.txt;
do
  echo "$i"
done

Or in a recursive manner (find also in all subdirectories):

for i in `find . -type f -name "*.img" -o -name "*.bin" -o -name "*.txt"`;
do
  echo "$i"
done
Benny
  • 2,233
  • 1
  • 22
  • 27
  • 1
    according to `man find`: `-o (meaning logical OR)` – Timo Jun 05 '21 at 19:49
  • 1
    I like the top solution of yours and upvoted it but I get an error if I do not have a file with that particular extension. Is there something I can do to ignore an extension if not found? – Matt Cremeens Jul 18 '21 at 12:50
14

the correct answer is @chepner's

EXT=java
for i in *.${EXT}; do
    ...
done

however, here's a small trick to check whether a filename has a given extensions:

EXT=java
for i in *; do
    if [ "${i}" != "${i%.${EXT}}" ];then
        echo "I do something with the file $i"
    fi
done
umläute
  • 28,885
  • 9
  • 68
  • 122
3

as @chepner says in his comment you are comparing $i to a fixed string.

To expand and rectify the situation you should use [[ ]] with the regex operator =~

eg:

for i in $(ls);do
    if [[ $i =~ .*\.java$ ]];then
        echo "I want to do something with the file $i"
    fi
done

the regex to the right of =~ is tested against the value of the left hand operator and should not be quoted, ( quoted will not error but will compare against a fixed string and so will most likely fail"

but @chepner 's answer above using glob is a much more efficient mechanism.

umläute
  • 28,885
  • 9
  • 68
  • 122
peteches
  • 3,447
  • 1
  • 13
  • 15
  • how would it be with a variable `ext` instead of `.java`? – AR89 Jan 24 '13 at 18:09
  • 2
    ack, no need for a regular expression: `if [[ $i == *.java ]]` or `if [[ $i == *.$ext ]]`. But [don't parse ls](http://mywiki.wooledge.org/ParsingLs) – glenn jackman Jan 24 '13 at 21:53
  • a beautiful solution, since I could do the search in subdirectories by simply using `for i in $(ls -lR); do ...`, or if you want the relative path of the file : `for i in $(find -L .);do ...` with the help of another good answer: https://stackoverflow.com/a/105249/6045793 – Mahdad Baghani Nov 02 '20 at 08:43
3

I agree withe the other answers regarding the correct way to loop through the files. However the OP asked:

The code above doesn't work, do you know why?

Yes!

An excellent article What is the difference between test, [ and [[ ?] explains in detail that among other differences, you cannot use expression matching or pattern matching within the test command (which is shorthand for [ )


Feature            new test [[    old test [           Example

Pattern matching    = (or ==)    (not available)    [[ $name = a* ]] || echo "name does not start with an 'a': $name"

Regular Expression     =~        (not available)    [[ $(date) =~ ^Fri\ ...\ 13 ]] && echo "It's Friday the 13th!"
matching

So this is the reason your script fails. If the OP is interested in an answer with the [[ syntax (which has the disadvantage of not being supported on as many platforms as the [ command), I would be happy to edit my answer to include it.

EDIT: Any protips for how to format the data in the answer as a table would be helpful!

user000001
  • 32,226
  • 12
  • 81
  • 108
3

I found this solution to be quite handy. It uses the -or option in find:

find . -name \*.tex -or -name "*.png" -or -name "*.pdf"

It will find the files with extension tex, png, and pdf.

Paul Rooney
  • 20,879
  • 9
  • 40
  • 61
f0nzie
  • 1,086
  • 14
  • 17