how to process files in a directory with bash shell

Question

Imagine that I have the directory as below:

dir
 |
 |--- sub1
 |     |
 |     |------filea.jpg
 |     |------fileb.jpg
 |
 |
 |--- sub2
       |
       |------filec.jpg
       |------filed.jpg
       |------sub21
                |
                |-------filee.jpg

What I need to do is to pass all of files which are located in the same directory as one parameter to some executable file.

For example, for the case above, I need to:

upload.sh dir/sub1/filea.jpg dir/sub1/fileb.jpg
upload.sh dir/sub2/filec.jpg dir/sub2/filed.jpg
upload.sh dir/sub2/sub21/filee.jpg

I'm not good at linux shell, what I have done is to list all of them with the script:

find dir -type f | while read myfile; do
    echo $myfile
done;

How to make a script for my need?

Do the filenames have a pattern that distinguishes them from subdirectories? — Barmar, Apr 29 '17 at 10:39
@Barmar Yes. All of filenames have an extension. For example, filea.jpg, fileb.jpg — Yves, Apr 29 '17 at 10:42
Possible duplicate of [How to iterate over files in a directory with Bash?](https://stackoverflow.com/q/20796200/608639), [Bash script to execute command on all files in a directory](https://stackoverflow.com/q/10523415/608639), [How to do something to every file in a directory using bash?](https://stackoverflow.com/q/1310422/608639), [for loop over specific files in a directory using Bash](https://stackoverflow.com/q/14823830/608639), etc. — jww, Aug 29 '18 at 15:05

Barmar · Accepted Answer · 2017-04-29T11:42:54.213

1

Use find to find the directories, then execute your command for all the files in the directory. Use the known extension in the wildcard to prevent it from including subdirectories.

shopt -s nullglob
find dir -type d | while read dir; do
    files=("$dir"/*.jpg)
    if [ ${#files[@]} -gt 0 ]
    then upload.sh "${files[@]}"
    fi
done

edited Apr 29 '17 at 11:42

answered Apr 29 '17 at 10:38

Barmar

741,623
53
500
612

It doesn't work for sub21. Meaning that it works only for the first level subdirectories. – Yves Apr 29 '17 at 11:10
Maybe the problem is that `upload.sh` doesn't work for subdirectories. – Barmar Apr 29 '17 at 11:18
It doesn't work if I remove the whole sub1 and sub2/filec and sub2/filed, meaning that there is only `dir/sub2/sub21/filee.jpg` I changed upload.sh as echo and I got the result including dir/*.jpg, I think this line should be removed. – Yves Apr 29 '17 at 11:23
That will get an error because it doesn't find any matching files. But it should still go on to process the other directories. Are you using `set -e` to abort the script on errors? – Barmar Apr 29 '17 at 11:33
When I use `echo`, I do see the result `dir/*.jpg dir/sub2/sub21/filee.jpg`. However, I think `upload.sh` can't accept `dir/*.jpg` so we must remove it. I can't change `upload.sh` because it is a binary coming from the Internet. – Yves Apr 29 '17 at 11:36
What does `upload.sh` do if it's given no arguments? You can use `set -o nullglob` so that a non-matching wildcard expands to nothing instead of staying as the wildcard. – Barmar Apr 29 '17 at 11:39
1

See http://stackoverflow.com/questions/91368/checking-from-shell-script-if-a-directory-contains-files for how to test if a directory contains any files that match the wildcard. You can put an `if` statement inside the loop. – Barmar Apr 29 '17 at 11:41
I've updated the answer to test if the directory contains any files. – Barmar Apr 29 '17 at 11:44

how to process files in a directory with bash shell

1 Answers1