0

I have a 100 MB log file, but I only need the last element (all the elements are separated by '\n'.), the structure of the file is

{JSON}\n{JSON}\n...{JSON}\n

And the goal is to get the last JSON unit.

How to do this efficiently?

Thansk!

yiping
  • 55
  • 1
  • 8
  • 1
    read in reverse. see: https://stackoverflow.com/questions/2301789/read-a-file-in-reverse-order-using-python – Ywapom May 23 '18 at 21:50

1 Answers1

0

I found a library called tailer (pip install tailer) which is similar to linux tail file_path --lines=1 command and it meets my goal.

import tailer
# Read the last n lines
tail = tailer.tail(open(file_path), last_n_lines))
# tail is a list of last n lines

Or, using the linechache package, which is able to get the line by line number. To get the last line, first to count the number of lines in a text file. 

total_line_number = sum(1 for line in open(file_path))

and then get the last line

import linecache
tail = linecache.getline(file_path, total_line_number)
yiping
  • 55
  • 1
  • 8