I am attempting to loop through the contents of a regular file which contains three things: 1) Description of the file 2) Naming convention of file in linux server 3) Linux server path
There will be about 20 or so lines with a similar format as below and if the file exists it will be put in to a csv file for a daily report.
Test File,TEST.$(date+'%Y-%m-%d'),/received/completed/$(date+'%Y-%m-%d')
There are multiple lines in similar format above with different file paths, file names etc.
The regular file is broken up into those 3 parts as described above, and I need to use a wildcard for todays date because thats the folder structure in the linux environment for completed files. Below is the code for reference. It runs throughout the day to check for file paths using the file above as an argument.
## Frequency in seconds to check the path
RECHECK_WAIT=5
## Start and end time to know when we should start
## and stop checking
START_TIME=0000
END_TIME=2359
LIST=$1
DATE_FORMAT=$(date +'%Y-%m-%d')
DATE_FORMAT1=$(date +'%Y%m%d')
REPORT_FILE="report_$(date +%b-%d-%y).csv"
check_if_report_exists() {
## Check to see if a report file already exists
## If not, use base template as starting point
if [[ ! -f $REPORT_FILE ]]
then
base_report=template.csv
else
base_report=$REPORT_FILE
fi
}
update_csv_file_recv() {
echo "$(date): Recieved "$2" in $LIST"
## Update .csv field for timestamp of recieved time and mark as Recieved
awk -v date="$(date +"%b-%d-%y %H:%M:%S")" -v type="$1" -v OFS=, -F, '{ if($2 ~ type) $3=date; if($2 ~ type) $6="Recieved"} 1' $base_report > tmp_$REPORT_FILE
cp tmp_$REPORT_FILE $REPORT_FILE 2>/dev/null
}
update_csv_file_processed(){
echo "$(date): "$2" was processed and no longer exists"
## Update .csv for timestamp of completed time and mark as Completed
awk -v date="$(date +"%b-%d-%y %H:%M:%S")" -v type="$1" -v OFS=, -F, '{ if($2 ~ type) $4=date; if($2 ~ type) $6="Completed"} 1' $base_report > tmp_$REPORT_FILE
cp tmp_$REPORT_FILE $REPORT_FILE 2>/dev/null
}
run_checks(){
## Check if file exists in path
check_if_exists="$(ls $1 | grep "$2")"
## Check .csv file to see if item is already marked as Received
check_already_recv="$(cat report_$(date +%b-%d-%y).csv 2>/dev/null | grep "$1" | awk -F, '{ print $6 }' | grep "Recieved")"
## Check .csv file to see if item is already marked as Completed
check_completed="$(cat report_$(date +%b-%d-%y).csv 2>/dev/null | grep "$1" | awk -F, '{ print $6 }' | grep "Completed")"
}
check_files() {
cat $LIST | while read line
do
TYPE="$(echo $line | awk -F, '{ print $1 }')"
EXPECTED="$(echo $line | awk -F, '{ print $2 }')"
path="$(echo $line | awk -F, '{ print $3 }')"
check_if_report_exists
## Check if files were recieved/processed
run_checks "$path" "$EXPECTED"
## If not tagged as completed in .csv perform
if [[ -z $check_completed ]]
then
## Check if file does not exist and not already recieved
if [[ ! -z $check_if_exists ]] && [[ -z $check_already_recv ]]
then
update_csv_file_recv "$TYPE" "$EXPECTED"
fi
## Check if file no longer exists, but was previously recieved
if [[ -z $check_if_exists ]] && [[ ! -z $check_already_recv ]]
then
update_csv_file_processed "$TYPE" "$EXPECTED"
fi
fi
done
## Wait a set amount of time before checking again
sleep $RECHECK_WAIT
## Update timestamp to see if we should no longer execute
check_time="$(date +%H%M)"
}
## Loop forever
while true
do
## Get current time
check_time="$(date +%H%M)"
## Check if current time is between our parameters
while [[ $check_time -ge $START_TIME ]] && [[ $check_time -lt $END_TIME ]]
do
### Main function used to execute checks and updates
check_files
done
### If its not within timeframe it will wait and try again in n-seconds
sleep $RECHECK_WAIT
done
I am having two issues here.
1) The awk command isnt reading the date as a variable as I want it to. The ls command in the check_files is not able to find the path (I know they exist for todays date). I believe its because the awk command isnt actually running the $(date) variable as it should.
ls: cannot access /done/$DATE_FORMAT: No such file or directory
2)Most recently i started getting this error for the time as well:
./file_exists.sh: line 75: [[: 0931: value too great for base (error token is "0931")
I run the script like this ./file_exist regular_text_file
Any advice or pointers would be appreciated!
