1

I have written a simple code which will send out an email when a service is down, once i restart the service,script will check the file for the same keyword. problem is it may find the earlier error in the log and give a false alarm that the service is still down. so I decided to search based on the time stamp.

dt=$(date +"%D %T")
awk '$0 ~ "Connection refused" && $0 >= $dt' /***.log

this is still returning all the old results as well

This is how the contents of the log look like.

[08/06/20 11:36:54.577]:Work...

Please let me know what I'm missing here and if this is the best way to go about with this. Edit: This is going to be an automated script that will be run every hour.

Thank you!

RavinderSingh13
  • 130,504
  • 14
  • 57
  • 93
user3581307
  • 23
  • 1
  • 5
  • 1
    Step 1. Convert the timestamp to seconds since epoch. 2. Compare as numbers. – KamilCuk Aug 06 '20 at 10:38
  • You need to check the last line containing 'Connection refused' and the desired timestamp. May be, you can save the timestamp of the last failure then compare it to the last line of the file containing the error after restart the service. For the last line, you can print it in the END{} section of AWK. – mchelabi Aug 06 '20 at 11:20
  • mchelabi, this is going to be an automated script so i will not be save the last failure and compare. – user3581307 Aug 06 '20 at 12:01
  • 1
    The current time will always be greater than the error time, you will never be able to find it in the file. Also, your AWK command is not correct, $td in your code doesn't contain the value of your variable. – mchelabi Aug 06 '20 at 12:20
  • While your along way off from a solution, it helps to focus readers attention on code that doesn't generate any warnings when checked at https://shellcheck.net . This Q gets asked at least 1x each month, did you try searching here for answers? Good luck. – shellter Aug 06 '20 at 14:14
  • Does this answer your question? [Filter log file entries based on date range](https://stackoverflow.com/questions/7706095/filter-log-file-entries-based-on-date-range) – shellter Aug 06 '20 at 14:30

1 Answers1

2

The reason you get the old results as well is that you don't really compare with that date, but with some undefined $dt inside the awk condition. The awk body is not a place where you use a bash variable as is. See how you do this: https://www.gnu.org/software/gawk/manual/html_node/Using-Shell-Variables.html

dt=$(date +"%D %T")
awk -v dt="$dt" '$0 >= dt && $0 ~ /Connection refused/' file

The alphabetical comparison seems enough for your case, I assume you look into logs of a few hours or days (I think that it could fail only around New Years Day, or not, depending maybe on the log file rotation and your environment).

To make it faster, as your log lines are still sorted by date, you want to search from the restart timestamp to the end of file, so you could set a flag when you find that timestamp and check for the pattern only after that:

awk -v dt="$dt" 'f && $0 ~ /Connection refused/{print; next} $0 >= dt {f=1}' file

You see that you don't check again any timestamps after the critical point. And in any case, it is better to match exactly the last service restart (how to do this depends on the details and you have not provided any) rather than comparing.

Edit: In the sample line of the question we have the timestamp inside brackets

[08/06/20 11:36:54.577]:Work...

and this can be passed e.g. with this modification

awk -v dt="$dt" 'f && $0 ~ /Connection refused/{print; next} substr($0,2) >= dt {f=1}' file

where substr($0,2) returns $0 without the first character.

thanasisp
  • 5,855
  • 3
  • 14
  • 31
  • $0 contains a string, you can't compare the date 'dt' with arithmetic operators '>='. Also, the supplied line is not complete and it may contain other character strings which may return false results. – mchelabi Aug 06 '20 at 12:53
  • 1
    @mchelabi `dt` is also a string (the output of a bash command) and this is alphabetical string comparison. – thanasisp Aug 06 '20 at 12:58
  • 2
    You just need to get past the `[` at the start of each line to get to the date for comparison. – Ed Morton Aug 06 '20 at 14:13
  • @thanasisp, yes I know, in this case, it's necessary to extract only the field containing the 'timestamp' then to compare it. – mchelabi Aug 06 '20 at 15:15