String Manipulation in Array of Strings with TR

Question

I am trying to automate the hashing of certain files (popular image and video formats) by using their file extension (not perfect but will suffice for my needs) recursively from the directory the script is run in.

I have little experience with bash and cannot figure out why this loop does not behave as intended although my suspicions are with the assignment of the F_EXTENSION variable and the | to tr command. It will hash all the files in the directory regardless of their extension. I am clearly doing something wrong.

The following is just a small part of the code for clarity, many more variables are assigned elsewhere (such as the filenames referenced below).

FILES=$( find ./* -type f )
EXTENSIONS=(.jpg .gif .png .bmp .avi .mpg .mov .mkv .flv .wmv .mp4)

SAVEIFS=$IFS
IFS=$'\n'
for FILE in $FILES; do
    F_EXTENSION=${FILE: -4} | tr '[:upper:]' '[:lower:]'
    if [[ "${EXTENSIONS[@]}" =~ "${F_EXTENSION}" ]]; then
        HASH=$( md5sum $FILE )
        echo ${HASH} | cut -c1-32 >> ${TEMPFILE}
    else
        echo "Skipping ${FILE}" >> ${LOGFILE}
    fi
done
IFS=$SAVEIFS

After trying various methods I have to concede its time for a fresh set of eyes. Any thoughts?

activated via `echo` in a subprocess and shielded by double quotes looks like a winning change, as in this "minimal" test case: `FILE='foo.MP3';x="$(echo ${FILE: -4}| tr '[:upper:]' '[:lower:]')"; echo $x` — Dilettant, Jun 18 '16 at 15:12
This might help: [How to debug a bash script?](http://unix.stackexchange.com/questions/155551/how-to-debug-a-bash-script) — Cyrus, Jun 18 '16 at 15:13
Dilettant, that does indeed solve the problem. Thank you very much! — user3931640, Jun 18 '16 at 19:09

score 2 · Answer 1 · answered Jun 18 '16 at 15:39

Your question is about

F_EXTENSION=${FILE: -4} | tr '[:upper:]' '[:lower:]'

This is really wrong in many respects. What it does is the following:

execute the command F_EXTENSION=${FILE: -4} in a subshell and then pipe its standard output to the command tr '[:upper:]' '[:lower:]'.

It's wrong since you certainly don't want the assignment F_EXTENSION=${FILE: -4} to be executed in a subshell (since all assignments made in a subshell are forgotten as soon as the subshell exits). Moreover, this assignment doesn't output anything to standard output, and hence tr never reads anything.

I understand you want to do the following: among all the files that have the extension given in the array EXTENSIONS and that are found (recursively) in the current directory, compute their md5sum, append this md5sum to file ${TEMPFILE}; among all the files that don't have this extension, output that you're skipping the file.

Here's a possibility to achieve that:

# Don't use an array to store the extensions, use a glob.
# We'll export it as it will be used in a child process (spawned by find)
# We'll also export the variables tempfile and logfile for the same reason
# To export them all automatically, we use set -a

set -a
extensions_glob=+(jpg|gif|png|bmp|avi|mpg|mov|mkv|flv|wmv|mp4)
tempfile=/path/to/tempfile
logfile=/path/to/logfile
set +a

find ./ -type f -exec bash -c -O nocasematch '
    if [[ $1 = *.$extensions_glob ]]; then
        md5sum < "$1" | cut -d" " -f1 >> "$tempfile"
    else
        echo "Skipping $1" >> "$logfile"
    fi
    ' bash {} \;

There would be other possibilities using Bash≥4, but you're stuck with Bash 3.2…

Since we're using globs, we don't need to use tr to force lowercase: we use the shell option nocasematch instead!

score 1 · Accepted Answer · edited May 23 '17 at 11:58

1

Aren't you intending to capture the output of the command?

F_EXTENSION="$(echo ${FILE: -4} | tr '[:upper:]' '[:lower:]')"

Right now your effectively assigning ${FILE: -4} to F_EXTENSION and then echoing nothing into tr. You need to use backticks or ${} to capture the stdout while also echoing the extension into tr.

If you'd like a more reliable way to get the extension without depending on its length, this offers this as a solution:

filename=$(basename "$fullfile")
extension="${filename##*.}"
filename="${filename%.*}"

edited May 23 '17 at 11:58

Community

1
1

answered Jun 18 '16 at 15:09

rrauenza

6,285
4
32
57

1

Hm on my OS X bash it yields bad substitution, when I use `"$(echo ...)" instead, it is ok. – Dilettant Jun 18 '16 at 15:14
3

Accidentally used {}'s -- needs to be ()'s. – rrauenza Jun 18 '16 at 15:24
Thanks for the solution, and I'll also implement the improved method for obtaining file extensions. – user3931640 Jun 18 '16 at 19:18

String Manipulation in Array of Strings with TR

2 Answers2