0

I have a log file, which holds the temperature values.
Using this code I can extract only temperature values from it.

Code:

import re
import itertools

infile = "/home/pi/Mysensor/Logs/test.log"
for line in open(infile):
    match = re.search('Temp=(\d+)', line)
    if match:
        test = match.group(1)
        print test

My log file:

2017-08-04 -> 16:14:29
Temp=28.0*  Humidity=36.0%

Code output:

28
28
25
29
28
25

What I want to do is, just extract only last four results.
I have tried with arrays and list. But could not get a result.

What am I missing here?
How to get this program to get only the final four result?

Thanks in advance.

hirosht
  • 932
  • 2
  • 17
  • 31
Sachith Muhandiram
  • 2,819
  • 10
  • 45
  • 94

4 Answers4

1

You can save the temperatures on a list, and use slices to get the 4 last:

import re
import itertools
temps = []
infile = "/home/pi/Mysensor/Logs/test.log"
for line in open(infile):
    match = re.search('Temp=(\d+)', line)
    if match:
      test = match.group(1)
      temps.append(test)
print temps[:-5:-1]

To see more about slices, see this post

Tisp
  • 436
  • 2
  • 15
1

I guess this really depends on how big your log file is, but there are a couple of ways I can think of to do it.

The easiest is probably to use a deque. 

from collections import deque
import re

temps = deque(maxlen=4)

infile = "/home/pi/Mysensor/Logs/test.log"
with open(infile, "r") as fh:
    for line in fh:
        match = re.search('Temp=(\d+)', line)
        if match:
            temp = match.group(1)
            temps.append(temp)
Batman
  • 8,571
  • 7
  • 41
  • 80
  • your answer iterates through, Tisp's answer is giving me the best soulution. – Sachith Muhandiram Aug 24 '17 at 13:48
  • They both iterate through. They're almost the same code. The only difference is that using a deque means you don't have to keep the whole array in memory. The downside is that you need to do an extra `pop` operation at each stage. Like I said, it depends on how big your file is. – Batman Aug 24 '17 at 13:54
1

One straightforward approach is to use tail from linux shell

  1 import os
  2 
  3 def my_tail(f, n):
  4     stdin, stdout = os.popen2("tail -n " + str(n) + " "+ f)
  5     lines = stdout.readlines();
  6     return lines
  7     
  8 print my_tail("./my_log.txt",4)
V K Shopov
  • 101
  • 1
  • 3
  • 1
    It's nice, but assumes that the user isn't on Windows box. – Batman Aug 24 '17 at 12:50
  • @VKShopov I m on linux, the output is `['Temp=18.0* Humidity=36.0%\n', '\n', '2017-08-04 -> 16:14:29\n', 'Temp=28.0* Humidity=36.0%\n']` , just the last line. – Sachith Muhandiram Aug 24 '17 at 13:43
  • @Sachith Well there are four '\n' - each of them counts as new line. So it seems that every record of Your data is consisted of several lines. – V K Shopov Aug 25 '17 at 06:41
0

If you are okay if some other language can do the same you may use the below commands in a bash shell (assuming the log file name is stack.log) :

grep 'Temp' stack.log | tail -4 | gawk -F= '{print $2}' | gawk '{print $1}' | sed s/*//g

Breaking the above commands :

  1. grep 'Temp' stack.log -> search for "Temp" string in given log file.
  2. tail -4 -> will extract last 4 records from above commands output.

  3. gawk -F= '{print $2}' -> uses "=" as delimiter and print the first column, for example : 29.0* Humidity 21.0* Humidity 22.0* Humidity 28.0* Humidity

  4. gawk '{print $1}' -> uses"space" as delimiter and print the first column alone, for example : 29.0*

    21.0*

    22.0*

    28.0*

  5. sed s///g -> replace all "" (asterisks) with blank "" (blank).

Final output looks like :

29.0

21.0

22.0

28.0

You may redirect this to a file and read in your program as temperatures.

Rudhin
  • 21
  • 1
  • 4