0

I am appending to a file in my Python code and reading the file again at a different time.
Example:

if not os.path.isfile(f'./test.rpt'):
    with open('test.rpt', 'a') as f:
        f.write(f'some string')

The size of this appended file can be millions of lines and I was only interested in the last line, so I am reading the file in binary mode so I can go to the last line as follows: Also, the last line is always 11 characters, thus the -12.

with open('test.rpt', 'rb') as f:
    f.seek(-12, 2)
    line = f.readline().decode("utf-8")
    last_line = line.rstrip('\n')
    print(f'Last line is: {last_line}')

But looks like my last line is always a new line. I am in read mode and still the last line looks to be a new line (as per the code, shouldn't the last line be 'some string'?). How can I avoid reading the new line here and read the last line that I had written to the file which was 'some string'.

When I manually open the file and write 'some string' to it, I am able to read it correctly when I read the file again using above code. It's an issue just when my script appends to the file. That makes me wonder if the write() function adds a new line character at the end of the file and if there is a way to avoid it? Or reading the second last line?

martineau
  • 119,623
  • 25
  • 170
  • 301
  • 2
    Why don't you just read your file in 'r' mode, then last_line = f.readlines()[-1] – Synthase Dec 30 '20 at 16:07
  • Thank you @Synthase. My understanding was readlines() will cause memory issues for huge files. –  Dec 30 '20 at 16:27
  • 1
    Rishabh: Computers usually have enough memory to handle files tens of millions of bytes long, so reading it all into memory would probably work — however it's unnecessary and potentially wastes a lot of time reading millions of bytes of data (since file I/O is often relatively slow) that are ignored. – martineau Dec 30 '20 at 16:40
  • Thank you. One thing I know for sure is that the file will be less than 100MB in size. –  Dec 30 '20 at 16:53
  • 1
    Note that `f.write(f'some string')` only writes **11** characters to the file, not 12 — a newline is not automatically added. Another thing to be aware of is the one some systems, line endings are two characters, `\r\n`, not one as your code assumes. This is usually hidden from you when reading files in text mode, but needs to be consider when doing so in binary mode. – martineau Dec 30 '20 at 16:54
  • 2
    It looks like your answer is here: https://stackoverflow.com/questions/3346430/what-is-the-most-efficient-way-to-get-first-and-last-line-of-a-text-file/18603065#18603065 – iqmaker Dec 30 '20 at 16:55

0 Answers0