Not able to loop through directories in a path stored in a variable

Question

I just don't get it, why I cannot do the thing I just mentioned in the question?

For more clarification, I am trying to loop through all the directories in a certain path, which is stored in any random variable of my choice. Please look at the below script and let me know why the execution doesn't get past the for loop beginning statement when I run the script:

gpll_allrepos3() {
    export SIMPROJS="/c/Projects/mainproject"
    currPath=$(pwd)
    for repoFolder in $SIMPROJS; do
        echo "RepoFolder: $repoFolder"
        if [[ -d "$SIMPROJS/$repoFolder/.git" && ! -L "$SIMPROJS/$repoFolder/.git" ]]; then
            echo "$SIMPROJS/$repoFolder"
            cd "$SIMPROJS/$repoFolder" && echo "Now inside $SIMPROJS/$repoFolder"
            curr_branch=$(git rev-parse --abbrev-ref HEAD)
            ## git pull && git pull origin
            ## if [[ "$curr_branch" != "master" ]]; then
                ### Merging master branch as the current branch is feature branch
                git checkout master
                git pull && git pull origin
                git checkout $curr_branch
                git merge master
            ## fi
        fi
    done
    cd $currPath
}
gpll_allrepos3

I am not a very good Bash scripter, so any suggestion is appreciated, but please note, I need to be able to run this script from anywhere I choose.

Maybe you want to look at the output of `for repoFolder in $SIMPROJS; do echo "$repoFolder"; done` and see if it's what you expect. — miken32, Nov 06 '18 at 21:18
I did that and it ony printed the parent path and that too, only once, strange, I thought it was gonna loop thro' sub-folders and not the parent path itself.. — Vicky Dev, Nov 06 '18 at 21:20
That's because you're not giving it anything to glob. Try adding `/*` to the end of your variable. — miken32, Nov 06 '18 at 21:27
Note: you don't need to save `curr_branch`, you can `git checkout -` (shorthand for `git checkout @{-1}`) to checkout the previous branch ref. https://stackoverflow.com/questions/7206801/is-there-any-way-to-git-checkout-previous-branch — masseyb, Nov 07 '18 at 01:00

Gordon Davisson · Accepted Answer · 2018-11-06T21:43:56.923

for doesn't iterate over the contents of directories, it iterates over a series of "words" (basically, strings). Thus, this statement:

for var in alice bob "jimmy joe" /c/Projects/mainproject; do

...will run its contents four times, first with var set to "alice", then "bob", then "jimmy joe", then "/c/Projects/mainproject". It doesn't treat any of those as file or directory names, because you didn't ask it to. On the other hand, the shell will expand /c/Projects/mainproject/* to a list of files in the directory /c/Projects/mainproject/ (including the path prefix). Thus, you could use:

for repoFolder in "$SIMPROJS"/*; do

The wildcard string will expand to a series of words (corresponding to the files and directories under /c/Projects/mainproject/), and then for will iterate over those.

Note the double-quotes around the variable reference? It's almost always a good idea to do this, otherwise the shell will split the variable's contents into separate words based on spaces (or whatever's in the IFS variable), and expand wildcards in the variable's value, and you usually don't want that (and if it happens unexpectedly, it can cause trouble). On the other hand, the * has to be outside the double-quotes so it'll expand to a list of files, not just be treated as a literal filename.

Also, note that this will iterate over files as well as subdirectories in /c/Projects/mainproject. If you want just subdirectories, use this:

for repoFolder in "$SIMPROJS"/*/; do

The added trailing slash will restrict it to just matching directories (because a slash at the end of a directory's path makes sense, but at the end of a plain file's path it doesn't). But note that the trailing slash will get included in the resulting paths as well.

Other notes: don't use "$SIMPROJS/$repoFolder" etc -- the repoFolder variable already includes the path prefix (because it was included in the wildcard string that got expanded), so adding it again will cause trouble. Just use "$repoFolder".

Also, when you use cd in a script, you should always include error handling in case it fails for some reason. Otherwise, the script will continue running in the wrong place, probably leading to chaos. You use:

cd "$SIMPROJS/$repoFolder" && echo "Now inside $SIMPROJS/$repoFolder"

...if this fails (which it will because of the path-doubling problem), it will not print the "Now inside..." message, but will then continue on with everything else. What you should use is something more like this:

cd "$repoFolder" || {
    echo "Couldn't cd into $repoFolder, cancelling..." >&2
    return 1
}

...which will exit the function rather than blindly continuing in the wrong place.

BTW, shellcheck.net is good at spotting common problems like these; I recommend running your scripts through it, at least until you get more familiar with shell scripting.

score 1 · Answer 2 · answered Nov 06 '18 at 21:38

To loop through files in a directory you need to provide a string that can be expanded to a filename glob.

Once you have that, your loop variable will contain the entire path, so no need to prefix the original directory name.

Also you can use pushd and popd to avoid creating your $currPath variable.

#!/bin/bash
simprojs="/c/Projects/mainproject"
for repoFolder in "$simprojs"/*; do
    if [[ -d "$repoFolder/.git" && ! -L "$repoFolder/.git" ]]; then
        pushd "$repoFolder" && echo "Now inside $repoFolder"
        curr_branch=$(git rev-parse --abbrev-ref HEAD)
        git checkout master
        git pull && git pull origin
        git checkout "$curr_branch"
        git merge master
    fi
done
popd

masseyb · Answer 3 · 2018-11-07T00:35:40.157

A script seems like overkill for the problem, can use find: find $SOMEPATH -type d -name .git -execdir git checkout master \; -execdir git pull && git pull origin \; -execdir ... \;.

find directories (-type d) named ".git" (-name .git) in $SOMEPATH. For each .git directory run command from the subdirectory containing the matched file (-execdir command \; -execdir command \; -execdir ... \;).

You don't need to cd to the .git directory, you don't need to maintain a stack of directories (e.g. pushd and popd), etc. One-liner. Note: by default find never follows symbolic links (ref. op's ! -L).

Not able to loop through directories in a path stored in a variable

3 Answers3