How to handle no matching files when using ${f%.gz} syntax

Question

I'm developing a script to process and move away compressed files that get dropped into a certain folder.

The script works perfectly as long as what gets dropped into the folder is a compressed file. However, if the script executes when there are no compressed files to process the bash function ${f%.gz} gives unexpected results.

Here is the script, with an example of the problem case afterwards:

FILES="$INGEST_DIR/*.gz"

for f in $FILES
do
    JUSTFILENAME=${f##/*/}
    syslog -s -l n "Archiving \"$JUSTFILENAME\""
    UNZIPPEDPATH=${f%.gz}
    syslog -s -l n "Moving \"$UNZIPPEDPATH\""
    UNZIPPEDNAME=${UNZIPPEDPATH##/*/}
    syslog -s -l n "    to \"ASR_DIR/$UNZIPPEDNAME\""
    syslog -s -l n "gunzip-ing $f"
    gunzip $f
    mv "$UNZIPPEDPATH" "$ASR_DIR/$UNZIPPEDNAME"
done

Again, it works perfectly if there's at least one .gz file in the target directory.

If there aren't any .gz, but there are other files in the directory (which must be there for other reasons) $FILES contains the expanded $INGEST_DIR plus the /*.gz, like this:

INGEST_DIR=/path/to/foo
FILES="$INGEST_DIR/*.gz"
echo $FILES

will show

/path/to/foo/*.gz

That isn't especially bad except that

for f in $FILES
do
    UNZIPPEDPATH=${f%.gz}
    echo $UNZIPPEDPATH
done

yields

somefile.txt someotherfile.exe yetsomeotherfile.dat

So is there an elegant way to not iterate if there are no such compressed files to handle? My script is working as well as it is because I just learned about ${f##/*/} and ${f%.gz} from this SO question & answer, so I'm thinking there might be a better way than

FILES="$INGEST_DIR/*.gz"

to start things off... or something to do right away before heading into the for loop.

I agree with [devnull](http://stackoverflow.com/questions/18100393/how-to-handle-no-matching-files-when-using-f-gz-syntax/18100521#18100521) that when it comes to files you should use power of `find` tool, as for `${foo%boo}` syntax, if it don't find `boo` in your `foo` var it will return `foo` unchanged. That syntax just tries to cut off `boo` from the end of the `foo` string. — rook, Aug 07 '13 at 10:25

score 2 · Answer 1 · answered Aug 07 '13 at 10:06

2

Use find for getting the list of files instead:

for f in $(find . -type f -name "*.gz"); do

done

For limiting the matches to the current directory, you could say:

find . -maxdepth 1 -type f -name "*.gz"

answered Aug 07 '13 at 10:06

devnull

118,548
33
236
227

Thank you @devnull, I like building the find right into the loop handler because it's clear what you're doing. – tobinjim Aug 07 '13 at 10:12
@devnull I think you set IFS to $'\n' as well to prevent splitting of filenames with spaces. Or if in Bash just use find . -type f -name "*.gz" | while IFS= read -r f; do ...; done – konsolebox Aug 07 '13 at 10:27
@konsolebox You could quote the `$(...)` and that would take care of *spaces* in filenames. – devnull Aug 07 '13 at 10:30
@devnull do you mean "$(...)"? but that would only be parsed as one string. – konsolebox Aug 07 '13 at 10:33
@konsolebox Did you try the quoted version? – devnull Aug 07 '13 at 11:01
The only thing that expands to multiple arguments are those of arrays e.g. "${ARRAY[@]}". "$(do something)" won't work. As an example this would only message "|a b cxyz|" not "|a b c||x y z|": for __ in "$(echo a b c; echo x y z)"; do echo "|$__|"; done – konsolebox Aug 07 '13 at 11:13

konsolebox · Accepted Answer · 2013-08-07T11:05:10.797

1

If you're using bash you could set nullglob before the loop:

shopt -s nullglob
for f in $FILES

---- Add ----

Another way to do it is by using while read and process substitution:

while IFS= read -r f; do
    ...
done < <(command)

Where command could be by the use of find or compgen, only that find could be more specific to files:

find -type f -name '*.gz'
compgen -G '*.gz'

If we're reading input within the loop, we could use other FDs when opening the file:

while IFS= read -ru 4 f; do
    ...
done 4< <(command)

edited Aug 07 '13 at 11:05

answered Aug 07 '13 at 10:05

konsolebox

72,135
12
99
105

2

Thank you konsolebox, that works nicely. I'm going to accept your answer to help with your rep :) even though I personally am going to use the answer @devnull gave since it will be easier for me to know what's going on when I revisit the script 6 months from now! – tobinjim Aug 07 '13 at 10:13
Welcome. I added more suggestions just in case they would be helpful as well. – konsolebox Aug 07 '13 at 11:06
They are helpful -- I'm still not fluent speaking in bash... in particular, compgen was new to me and looks handy. http://www.cyberciti.biz/open-source/command-line-hacks/compgen-linux-command/ <-- nice article, even for us Mac OS X users ;) – tobinjim Aug 07 '13 at 15:30

How to handle no matching files when using ${f%.gz} syntax

2 Answers2