1

My logs are in this format:

[2018-01-22T13:40:22,825][WARN ] message
[2018-01-22T13:41:52,830][ERROR ] message
[2018-01-22T13:45:27,831][WARN ] message

I need to write a script that will check to see if there have been any errors in the last 5 minutes. From what I've found online, something like this should work:

awk -v d1="$(date --date="-5 min" "+%Y-%m-%dT%H:%M:%S")" -v d2="$(date "+%Y-%m-%dT%H:%M:%S")" '$0 > d1 && $0 < d2 || $0 ~ d2' log.txt

But it doesn't. I think that the [ is getting in the way maybe? What else can I try?

tannerwj
  • 154
  • 3
  • 11
  • Did u already saw this https://stackoverflow.com/questions/20649387/extract-last-10-minutes-from-logfile ? – Black.Jack Jan 22 '18 at 22:36
  • @Black.Jack Yes, that's where I got my start on. But my format is different and I can't get it to work – tannerwj Jan 22 '18 at 22:37

2 Answers2

2

Awk solution:

Sample log.txt:

[2018-01-22T13:40:22,825][WARN ] message
[2018-01-23T00:38:37,830][ERROR ] message
[2018-01-22T13:45:27,831][WARN ] message

awk -v d1="$(date --date="-5 min" "+%Y-%m-%dT%H:%M:%S")" \
    -v d2="$(date "+%Y-%m-%dT%H:%M:%S")" \
    '$2 > d1 && $2 < d2 && $4~/ERROR/' FS='[\\[\\]]' log.txt
  • FS='[\\[\\]]' - treat square brackets as complex field separator

The output:

[2018-01-23T00:38:37,830][ERROR ] message
RomanPerekhrest
  • 88,541
  • 4
  • 65
  • 105
  • Wow, that's more than I thought possible with awk. I was just going to pipe that into grep and look for the errors I was watching for. I can modify this though and it'll help clean things up – tannerwj Jan 22 '18 at 23:03
  • `that's more than I thought possible with awk` - then you have a significant misunderstanding of what awk is :-)! – Ed Morton Jan 22 '18 at 23:30
2

You're right, [ is getting in the way.

The lexicographical compare, $0 > d1, compares something like:

[2018-01-22T13:45:27,831][WARN ] message

with

2018-01-22T13:45:27

and [... is always greater than any string starting with a number, e.g. 20...

Similarly for $0 < d2 -- but in this case the condition is never satisfied ([20.. is always greater than 20..). That's why you're not getting any output.

A quick'n'dirty fix is to simply format d1 and d2 to start with [:

awk -v d1="$(date --date="-5 min" "+[%Y-%m-%dT%H:%M:%S")" -v d2="$(date "+[%Y-%m-%dT%H:%M:%S")" '$0 > d1 && $0 < d2' log
randomir
  • 17,989
  • 1
  • 40
  • 55
  • 1
    Worked great, thanks. I had tried adding the [ but I did it before the +... – tannerwj Jan 22 '18 at 23:02
  • 1
    @tannerwj you could also concatenation it in ask I think - just ‘$0 > “[“ d1’ for example in the pattern. – Guy Jan 23 '18 at 01:15