Multiple elements instead of one in bash script for loop

Question

I have been following the answers given in these questions

Shellscript Looping Through All Files in a Folder

How to iterate over files in a directory with Bash?

to write a bash script which goes over files inside a folder and processes them. So, here is the code I have:

#!/bin/bash

YEAR="2002/"
INFOLDER="/local/data/datasets/Convergence/"

for f in "$INFOLDER$YEAR*.mdb";
do
    echo $f
    absname=$INFOLDER$YEAR$(basename $f)

    # ... the rest of the script ...

done

I am receiving this error: basename: extra operand.

I added echo $f and I realized that f contains all the filenames separated by space. But I expected to get one at a time. What could be the problem here?

Inian · Answer 1 · 2017-09-06T17:22:18.423

Just quote your shell variables if they are supposed to contain strings with spaces in between.

basename "$f"

Not doing so will lead to splitting of the string into separate characters (see WordSplitting in bash), thereby messing up the basename command which expects one string argument rather than multiple.

Also it would be a wise to include the * outside the double-quotes as shell globbing wouldn't work inside them (single or double-quote).

#!/bin/bash

# good practice to lower-case variable names to distinguish them from
# shell environment variables

year="2002/"
in_folder="/local/data/datasets/Convergence/"

for file in "${in_folder}${year}"*.mdb; do
    # break the loop gracefully if no files are found
    [ -e "$file" ] || continue 
    echo "$file"
    # Worth noting here, the $file returns the name of the file
    # with absolute path just as below. You don't need to 
    # construct in manually
    absname=${in_folder}${year}$(basename "$file")
done

glob would expand to nothing only when `shopt -s nullglob` is set and there are no matching files. So, I think Inian's `[ -e "$file" ]` is valid. — codeforester, Sep 06 '17 at 17:04

score 2 · Accepted Answer · answered Sep 06 '17 at 17:03

You're running into problems with quoting. In the shell, double-quotes prevent word splitting and wildcard expansion; generally, you don't want these things to happen to variable's values, so you should double-quote variable references. But when you have something that should be word-split or wildcard-expanded, it cannot be double-quoted. In your for statement, you have the entire file pattern in double-quotes:

for f in "$INFOLDER$YEAR*.mdb";

...which prevents word-splitting and wildcard expansion on the variables' values (good) but also prevents it on the * which you need expanded (that's the point of the loop). So you need to quote selectively, with the variables inside quotes and the wildcard outside them:

for f in "$INFOLDER$YEAR"*.mdb;

And then inside the loop, you should double-quote the references to $f in case any filenames contain whitespace or wildcards (which are completely legal in filenames):

echo "$f"
absname="$INFOLDER$YEAR$(basename "$f")"

(Note: the double-quotes around the assignment to absname aren't actually needed -- the right side of an assignment is one of the few places in the shell where it's safe to skip them -- but IMO it's easier and safer to just double-quote all variable references and $( ) expressions than to try to keep track of where it's safe and where it's not.)

score 0 · Answer 3 · answered Sep 06 '17 at 17:00

0

just remove "" from this line for f in "$INFOLDER$YEAR*.mdb";

so it looks like this

#!/bin/bash

YEAR="2002/"
INFOLDER="/local/data/datasets/Convergence/"

for f in $INFOLDER$YEAR*.mdb;
do
    echo $f
    absname=$INFOLDER$YEAR$(basename $f)

    # ... the rest of the script ...

done

answered Sep 06 '17 at 17:00

Hani

1,354
10
20

This is a hypercorrection; you correctly unquote the `*`, but *incorrectly* unquote the parameter expansions. – chepner Sep 06 '17 at 17:12

Multiple elements instead of one in bash script for loop

3 Answers3