How to get last error string based on the time it was printed?

Question

I have a log file which is auto updated and looks like this:

...
[23:32:19.586] PULL START
[23:32:19.637] PULL RESP NONE
[23:32:22.576] Rx - +CMS ERROR: 29
[23:32:22.686] STAT - TRY 2
[23:32:22.797] Tx - AT+CMGF=1
[23:32:23.008] Rx - OK
[23:32:23.017] Tx - at+cmgs="number"
[23:32:23.428] Rx - >
[23:32:23.438] Tx - message
[23:32:24.675] PULL START
[23:32:24.714] PULL RESP NONE
[23:32:26.663] Rx - +CMS ERROR: 29
[23:32:26.681] STAT - 68$$"+CMS ERROR: 29"
[23:32:26.695] SEND - RESPONSE, TRANS ID = xxxxxxxx, RESP CODE = xx, MESSAGE = +CMS ERROR: 29

and I have a list to be compared which is looks like this:

[
    '+CMS ERROR: 8',
    '+CMS ERROR: 28',
    '+CMS ERROR: 29',
    '+CMS ERROR: 50',
    '+CMS ERROR: 226',
]

All I want to do is if the last line of the log file has string +CMS ERROR: XX and matches one from the list, I want to terminate the log related program.

Note that the log file will keep updating at random time as long as the program is running and my program will re-check the log file each seconds. If the updates(the last line printed on the log file) does not contain any of the string on the list, it will not terminate any program.

Is it possible to do that in python? like using regex or something? Please help.

What OS are you on? This seems considerably easier to implement with a shell script and `cron`. — Adam Smith, May 09 '19 at 01:52

score 0 · Accepted Answer · answered May 09 '19 at 02:23

So there's three major parts of this script.

Read and parse a log
Conditionally kill a process
Repeat every x seconds

The first part is easy. Let's call it should_act.

def should_act():
    errors = ['+CMS ERROR: 8',
        '+CMS ERROR: 28',
        '+CMS ERROR: 29',
        '+CMS ERROR: 50',
        '+CMS ERROR: 226']

    with open("path/to/logfile.log") as f:
        for line in f:
            pass

    return any(error in line for error in errors)

The second part isn't too bad either. Let's call that act.

def act():
    pid = YOUR_PROCESS_ID
    subprocess.run(['taskkill', '/PID', str(pid)])
    # or alternatively taskkill /IM YOUR_IMAGE_NAME works too.

The third part creates some problems, but ultimately isn't too bad either. There are lots of ways to do this, the best being to schedule it outside of the application. taskschd.msc is the best way to do this on Windows, and cron is the best way in general.

Doing this in application has a bunch of answers, some better than others. I'll let you select from those solutions and instead advise that you use the OS to schedule the script to run every x seconds.

import subprocess

# the two code blocks above

if __name__ == "__main__":
    if should_act():
        act()

score 0 · Answer 2 · answered May 09 '19 at 02:31

You want to continuously watch the file? Exactly akin to what the Unix command tail -f does? Then before a code pointer, I suggest picking the right tool for the job. Outsource. If you want an in-process solution, take a look at Watchdog. If you are comfortable reading from a subprocess, consider any of the solutions in A Windows equivalent of the Unix tail command

Meanwhile, if you absolutely must open the file afresh each time, seek to the end first for efficiency:

with open('mylog.txt') as logf:
  logf.seek(-1024, 2)    # 2 = magic number to say "from end of file"
  last_line = logf.readlines()[-1]
  for exit_error in exit_error_strings:
    if exit_error in last_line:
      raise SystemExit    # just exit

Now, this assumes that no log line will be more than 1024 characters. If that's not a safe assumption, then obviously choose a value that is safe or add additional logic as appropriate.

Regarding regular expressions, they are often more expensive (computationally, memory) than you think, but if you've measured, you could also do something like:

import re

exit_error_re = re.compile(r'\+CMS ERROR: \d\d')
...

if exit_error_re.search(last_line):
  # do something

Obviously, set the regex as appropriate for your needs.

MrKioZ · Answer 3 · 2019-05-09T02:18:50.463

-1

you can perform this action by using converting your file into a list of an array to get the last line of the file. you can put it in a loop so it can get updated automatically I choose for this example Error number 8

from os import stat

filename = 'log.txt'

statinfo = os.stat(filename)

size = int(str(statinfo.st_size).replace('L', ''))

with open(filename, 'r') as f:
    array_list = fin.seek(size/2) #the will read half of the incase the file size is and you want fast way to read your file
    array_list = array_list.readlines()
    if '+CMS ERROR: 8' in array_list[len(array_list)-1]:
        #Your Code Here

edited May 09 '19 at 02:18

answered May 09 '19 at 01:56

MrKioZ

515
4
14

no reason to do `f.readlines` here, especially since this file sounds like it will get very long. Doubly no reason to do `f.close` since as soon as you leave the `with` block it will close anyway. – Adam Smith May 09 '19 at 01:58
I agree with @AdamSmith, it will keep updating until the new file is created in the 00:00:000 AM in the next day. – Yura May 09 '19 at 02:01
@Adam4HD you've created a SyntaxError here (you forgot to close your `int(...)`) but this is not a great approach regardless. Why read half the file? – Adam Smith May 09 '19 at 02:13
@AdamSmith to make readlines run faster as we don't need the rest of the file – MrKioZ May 09 '19 at 02:18

How to get last error string based on the time it was printed?

3 Answers3