Bash for cycle with multiple conditions

Question

I'm practically new in bash scripting and still in learning process but I have trying to achieve relatively simple task but beyond my understanding.

I have output file with identifier, name of the machine and configured time of the MySQL dump which look like:

id;server.name;time_in_hour

for every single line of the file.

What I need is for loop which compares "time_in_hour" with timestamp of files in "server.name" folder on backup machine. Path to the backup files looks like:

/dumps/server.name/2020-05-06___10-49-00___sys.sql.gz

If there isn't backup files of max. time length 2x "time_in_hour" (example: actual time -4 hours, for every dump file in folder with 2 hours backup schedule) or if there isn't backup at all, than script will print error to output. In ideal case second condition of the script check end of every dump file for consistency, something like grep string "Dump completed".

End of a dump file example:

/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS /; /!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION /; /!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2020-03-02 23:58:29

I'm still not completely sure which tools is better for accomplishing the task "date" or "find"? Or how to practically build this type of ?nested loop? with multiple condition... can you please provide me some example?

Add a few lines of your mysql dump to your question (no comment). — Cyrus, May 06 '20 at 17:10

U880D · Accepted Answer · 2020-05-12T10:46:28.477

I'm still not completely sure which tools is better for accomplishing the task date or find?

This will somehow depend on your requirements and what you try to achieve. To i.e. just create a list of servers which do not have an actual backup you could use something like

find dumps/* -mmin +120 -type f | cut - "/" -f 2 > noBackup.lst

Since I had a somehow similar requirement in one of my environments I like to show some possible solution approaches.

toBackup.lst

1;test1.example.com;2
2;test2.example.com;2
3;test3.example.com;2

Since I haven't understand fully the what exactly is meant by time_in_hour I'll leave the value as given from your description

If there isn't backup files of max. time length 2x time_in_hour (example: actual time -4 hours, for every dump file in folder with 2 hours backup schedule) ...

as I think that there should be a backup have been made every 2hrs.

testBackup.sh

#!/bin/bash

BOLD='\033[0;1m'
RED='\033[0;31m'
YELLOW='\033[0;33m'
GREEN='\033[0;32m'
NC='\033[0m' # No Color

while read LINE; # from a list
do

  ID=$(echo "${LINE}" | cut -d ";" -f 1)
  SERVERNAME=$(echo "${LINE}" | cut -d ";" -f 2)
  TIMESLOT=$(echo "${LINE}" | cut -d ";" -f 3) # in hr
  TIMESLOT=$((${TIMESLOT}*60*60)) # in sec

  echo -e "${BOLD}ID ${ID}${NC}"
  echo "Going to check backup of ${SERVERNAME}"

  # Suppress error message if there is no directory or file
  FILENAME=$(ls 2> /dev/null dumps/${SERVERNAME})

  if [[ -z "${FILENAME}" ]]; then
     echo -e "${RED}No file found${NC}"
     # Since there is no backup file leave the loop and continue with next line from list
     continue
  else
     echo "Backup file is ${FILENAME}"
  fi

  # Get backup timestamp from filename instead of modification time of the file itself
  # To do so, cut out the timestamp and re-format it for further processing with date
  TIMESTAMP=$(echo "${FILENAME}" |  cut -d "_" -f 1-4 | sed 's/___/ /g' | sed 's/-/:/3g')
  echo "Backup time is ${TIMESTAMP}"

  # To be able to calculate the difference, convert timestamp and time to seconds 
  MTIME=$(date -d "${TIMESTAMP}" +"%s") # modification time
  NOW=$(date +"%s")
  DIFFERENCE=$((${NOW}-${MTIME}))

  # Check if difference not more than the given ${TIMESLOT} value
  echo "Allowed is an backup file age of ${TIMESLOT} sec"

  if [ ${DIFFERENCE} -gt ${TIMESLOT} ]; then
      echo -e "Backup file age of ${DIFFERENCE} sec is ${RED}too old${NC}"
      # Since the backup filename indicates it is too old, leave the loop and continue with next line from list
      continue
  else
      echo -e "Backup file age of ${DIFFERENCE} sec is ${GREEN}OK${NC}"
  fi

  # Check if the backup file contains a valid backup
  # To do so, decompress the file and gather the timestamp from last line
  DUMPTIME=$(gzip --decompress --to-stdout dumps/${SERVERNAME}/${FILENAME} | tail -1 | cut -d " " -f 6-)
  echo "Timestamp of database dump is ${DUMPTIME}"

  # Convert timestamp from within the backup file into seconds
  DTIME=$(date -d "${DUMPTIME}" +"%s")
  DIFFERENCE=$((${NOW}-${DTIME}))

  # Here it is assumed that the decompression worked well and the file contained an timestamp
  if [ ${DIFFERENCE} -gt ${TIMESLOT} ]; then
      echo -e "Backup content age of ${DIFFERENCE} sec is ${RED}too old${NC}"
      # Since the timestamp within the backup indicates it is too old, leave the loop and continue with next line from list
      continue
  else
      echo -e "Backup content age of ${DIFFERENCE} sec is ${GREEN}OK${NC}"
  fi

done < toBackup.lst

This will iterate over the given list of hosts, check if there are backup files, as well check if the file are valid and not too old. You may change or extend it for your needs.

Thanks to

Thank you kindly! This is exactly what I'm trying to achieve. — Jan Ves., May 13 '20 at 08:51

Bash for cycle with multiple conditions

1 Answers1