How to accurately grab logs in the past hour from a log file in BASH?

Question

I have a log file for cron jobs with entries using the format of

Mar  8 17:30:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)

I want to be able to periodically checks the logs to see if a specific CRON job ( that runs hoursly) ran. Finding the logs I want have been easy via grep.

grep name_of_job logfile

But I'm unsure of how to accurately grab only the logs from the past hour to see if any cron jobs have ran.

I tried using the awk command to grab the last hour based on this post but I've had issues with awk because my datestamp is separated into multiple fields.

score 2 · Answer 1 · answered Mar 09 '16 at 01:06

2

You can do this in GNU awk:

$ cat tst.awk
BEGIN {
    FS="[ :]+"
    nowSecs  = systime()
    thisYear = strftime("%Y")
}
{
    mthNr = (match("JanFebMarAprMayJunJulAugSepOctNovDec",$1)+2)/3
    inSecs = mktime(thisYear" "mthNr" "$2" "$3" "$4" "$5)
}
(nowSecs - inSecs) <= 60
$ awk -f tst.awk file

If your input file and the current date can cross year boundaries then you'd need to add some logic to handle it since your input file doesn't contain a year.

answered Mar 09 '16 at 01:06

Ed Morton

188,023
17
78
185

Thanks I already implemented the other guys because it worked and he posted first but I like your answer and like the little trick you did with the long month string :). – devinov Mar 23 '16 at 20:50
Thanks. You might want to read http://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice before trying to use a shell loop to manipulate text. – Ed Morton Mar 23 '16 at 23:18

riteshtch · Accepted Answer · 2016-03-09T00:17:42.803

wrt your problem I have created dummy logs, also note that time is wrt to my timezone. I have also displayed the current time before running the script.

$ cat file2
Mar  9 05:20:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
Mar  9 04:20:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
Mar  9 05:35:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
Mar  5 05:35:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
Mar  9 04:35:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
$ date
Wed Mar  9 05:40:10 IST 2016
$ ./script.bash 
Mar  9 05:20:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
Mar  9 05:35:01 hostname CROND[PROC_#]: (user) CMD (/path/to/job/)
$

Contents of script.bash are as follows:

#!/bin/bash

while read -r line || [[ -n "$line" ]]
do
    log_date_str="$(awk '{print $1" "$2" "$3}' <<< "$line")"
    log_date="$(date -d "$log_date_str" +%s)"
    [[ $(($(date +%s)-$log_date)) -le 3600 ]] && echo "$line"
done < "file2"

we get the date from the log file and convert it into number of seconds elapsed since Unix Epoch and also get that same format for current datetime from command date +%s Note +%s is the format here. date -d dateString +%s can be used to parse a string as date into a desired format.

Once we have both the dates we just subtract it and then find out those results which have difference less than 3600 seconds(1 hour); and then we just print out the log line.

Instead of passing the filename to the while loop you can pass your ouput of grep

Thanks, it works great, can you explain what the "while read..." line does? — devinov, Mar 09 '16 at 02:24
it reads the file line by line, using the `read` shell builtin, `help read` for more info. in short it reads each line of file into a variable called `line`(successfully read returns 0 exit code) or also checks if length of `line` variable is not 0 using `[[ -n line]]`(this is needed when line doesnt end in a newline char(`\n`), for example: last line) — riteshtch, Mar 09 '16 at 05:25

How to accurately grab logs in the past hour from a log file in BASH?

2 Answers2

Linked