0

I am trying to write a script which will check subdirectories for the existence of a compressed file, and if file does not exist, compress necessary file.

I have tried two versions of my script:

VERSION 1

for directory in /pool/folder/ID*/; do
    echo "Starting for loop"
    if [ -e /pool/folder/ID*/*.txt.gz ]
    then
        echo "file already exists"
    else
        echo "File does not exist"
        echo "file is being compressed"
        gzip *.txt
    fi
done

This didn't work, and I assumed it had something to do with my -e flag: http://www.ducea.com/2009/03/05/bash-tips-if-e-wildcard-file-check-too-many-arguments/

Once my "if" statement starts, the terminal output looks like it's trying to check every subdirectory on my computer:

Display all 4854 possibilities? (y or n)

And then it proceeds to list every directory. So I tried removing my -e flag and using ls:

VERSION 2

for directory in /pool/folder/ID*/; do
    echo "Starting for loop"
    files=$(ls /pool/folder/ID*/*.txt.gz 2> /dev/null | wc -l)
    if [ **"$files" != "0"** ]
    then
        echo "zipped file already exists"
    else
        echo "Zipped file does not exist"
    fi
done

But the same thing happened (listing out all the directories).

J. Fox
  • 33
  • 1
  • 1
  • 6
  • Globbing patterns don't expand in tests `[ -e /pool/folder/ID*/*.txt.gz ]`. You need to iterate the expansion and test each individual entry. – Léa Gris Jan 13 '20 at 14:56
  • 1
    @LéaGris they do expand, the problem is that the `-e` operator expects a single operand, while the glob might expand to 0 or multiple words – Aaron Jan 13 '20 at 15:00
  • @J.Fox: Why do you loop over the directories and set the variable `directory` to the directory you are focussing in the respective iteration, if you then don't use this variable? – user1934428 Jan 13 '20 at 15:02
  • Why not just `for i in /pool/folder/ID*/*.txt*; do [[ $i =~ ^.*[.]gz ]] || gzip "$i"; done`? Or perhaps even `for i in /pool/folder/ID*/*.txt; do gzip "$i"; done`? – David C. Rankin Jan 13 '20 at 15:08
  • @user1934428, I was originally using the variable directory in previous versions, but I took it out in the version I posted. Sorry about the confusion – J. Fox Jan 13 '20 at 15:26
  • @DavidC.Rankin, I need to check if the zipped files already exist, and skip over zipping the .txt file if they do (sometimes both exist in the same folder--once all files are zipped, I'll have to remove extraneous .txt files). I need to avoid creating duplicate zipped files for a future script I have to run. – J. Fox Jan 13 '20 at 15:29
  • Maybe, but this is what is killing you now. Not using this variable, lured you into using a `*` wildcard in your `if` statement, which is breaking your neck. – user1934428 Jan 13 '20 at 15:32
  • @J.Fox then `(for i in /pool/folder/ID*/*.txt; do [ -e "$i".gz ] || gzip "$i"; done)` will work. – David C. Rankin Jan 13 '20 at 16:32

1 Answers1

0

Is this what you are after? This will enter directory dir and check all files in there. If the file is already gziped, move on, if the file is not gziped, zip it.

for dir in /pool/folder/ID*; do
    [ -d "${dir}" ] || continue
    printf "processing %s\n" "${dir}"
    for file in "${dir}"/*.txt; do
       if [ -f "${file}.gz" ]; then
          printf "%s already compressed\n" "${file}"
       elif [ -f "${file}" ]; then
          printf "%s being compressed\n" "${file}"
          gzip "${file}"
       fi
    done
done

A bit shorter would be:

for dir in /pool/folder/ID*; do
    [ -d "${dir}" ] || continue
    printf "processing %s\n" "${dir}"
    for file in "${dir}"/*.txt; do
       [ -f "${file}.gz" ] && printf "%s already compressed\n" "${file}" && continue
       [ -f "${file}" ] && printf "%s being compressed\n" "${file}" && gzip "${file}"
    done
done
kvantour
  • 25,269
  • 4
  • 47
  • 72
  • This worked great! However, I am confused as to why it worked with a "//" in the file path: processing /pool/folder/ID123/ /pool/folder/ID123 **//** ID123.txt being compressed – J. Fox Jan 14 '20 at 21:25
  • The second slash comes from the second for loop. It adds an extra slash after the directory. You could remove the slash in the for loop of the directories and then your issue is solved – kvantour Jan 14 '20 at 22:00
  • I did remove the extra slash and it worked great! But it also worked great with the accidental double //, which I was not expecting. I was just curious as to why // didn't throw an error with the file path, if you happened to know. Just would have expected an issue with it, but there was none. – J. Fox Jan 15 '20 at 13:07
  • @J.Fox A double slash is not a issue unless the path starts with it. [ See [here](https://stackoverflow.com/questions/20690828/what-does-double-slash-in-cd-mean-in-linux) ] – kvantour Jan 15 '20 at 13:09