1

I am trying to get last line from a text file, which I used the solution from What is the most efficient way to get first and last line of a text file?

def read_last_line(filename):
    with open(filename, "rb") as f:
        first = f.readline()
        if f.read(1) == '':
            return first
        f.seek(-2, 2)  # Jump to the second last byte.
        while f.read(1) != b"\n":  # Until EOL is found...
            f.seek(-2, 1)  # ...jump back the read byte plus one more.
        last = f.readline()  # Read last line.
        return last.decode('ascii')

It managed to get the last line of the text file successfully if the file is modified by another script/program, but when I modify the text file using Notepad++ the exact same modification as another script/program, it will throw the following exception:

in read_last_line
    f.seek(-2, 2)
OSError: [Errno 22] Invalid argument

What I am trying to do is, I used watchdog to check if there's file changes, and on modify I will call read_last_line on the modified file.

Sample file

11/26/2020 2:05:12 PM Time Updated: +2ms            Regular Update
11/26/2020 2:06:13 PM Time Updated: +4ms            Regular Update
11/26/2020 2:07:13 PM Time Updated: +1ms            Regular Update

How I am calling the function:

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import ntpath

class FileEventHandler(FileSystemEventHandler):
    def __init__(self, filetowatch):
        self.file = filetowatch
    
    def on_modified(self, event):
        modified_file = ntpath.basename(event.src_path)
        if modified_file == self.file:
            read_last_line(event.src_path)

if __name__ == "__main__":
    event_handler = FileEventHandler("sample.txt")
    observer = Observer()
    observer.schedule(event_handler, path='C:/Program Files (x86)/SomeTime', recursive=False)
    observer.start()

May I know if anyone know what is causing the error?

Platform: Windows 10, Python 3.7.4

UPDATE - ANSWER

So the error was because of fread(1) == '' which was fixed using falsetru's solution.

The reason why it is not performing the way I expected was because text editor deletes the sample file and create a file using the same file name, therefore fread(1)=='' was triggered (which throws) and using script/program to modify the sample file did not throw simply because I did not delete the file.

  • 2
    Is it possible that Notepad++ is deleting the file when you save and creating another one with the same name? – Mark Ransom Nov 26 '20 at 04:54
  • @MarkRansom it also crashed using sublime and notepad as well.. I will add in code to check if file deletion is triggered. – Potato is my name Nov 26 '20 at 04:58
  • @MarkRansom indeed, text editor is deleting my file, which caused it to throw error when I do `f.read(1) == ''` - not using binary string caused the error. – Potato is my name Nov 26 '20 at 05:50

1 Answers1

1

If there's only a single (with / without trailing newline), while loop condition will never be met.

That cause f.seek(-2, 1) to try to seek to negative file position which causes the error.

Guard such case (prevent trying to seek beyond the file beginning), using .tell() which let you know current file position:

        while f.tell() >= 1 and f.read(1) != b"\n":

or you can use seek(..) return value:

        while f.read(1) != b"\n":
            if f.seek(-2, 1) == 0:
                break  # prevent going beyond file beginning.

UPDATE

In binary mode <io object>.read() return bytes object. In if condition, the code is comparing the bytes object with a string ''; which will always fail because of type difference. Changing to compare with bytes literal will fix the issue.

        if f.read(1) == b'':
            return first
falsetru
  • 357,413
  • 63
  • 732
  • 636
  • this is does not solve the issue..it did not throw error because the file position is wrong. the exact same modification i made and output to the file using a script will not cause an error and will return the correct last line. – Potato is my name Nov 26 '20 at 04:54
  • @Potatoismyname, https://asciinema.org/a/RYBfGjob1mJuxIVixVfE208Wa – falsetru Nov 26 '20 at 04:58
  • thanks for the video, but i have tried both solution you've provided, it is still the same issue. No error if it's modified through script/program/command prompt (I am using Windows) but error when it's modified through Notepad++/sublime/notepad .. – Potato is my name Nov 26 '20 at 05:04
  • @Potatoismyname, Could you provide a file you used. And code lines that call the function `read_last_line`? – falsetru Nov 26 '20 at 05:06
  • I have updated the question with a sample file. I just copy and paste the same line to the bottom of the file using text editor. – Potato is my name Nov 26 '20 at 05:10
  • @Potatoismyname, I updated the answer. In short, replace `''` with `b''`. – falsetru Nov 26 '20 at 05:17
  • I have also added the code snippet that i am calling `read_last_line` with. – Potato is my name Nov 26 '20 at 05:21
  • I have added `b''` but it does not solve the issue. – Potato is my name Nov 26 '20 at 05:22
  • @Potatoismyname, Could you add `print(repr(first_line), f.tell())` line before `f.seek(-2, 2)` line and let me know the output? – falsetru Nov 26 '20 at 05:29
  • @Potatoismyname, As Mark Ransom commented, it could be possible notepad delete/modify while `read_last_line`. Try `time.sleep(0.5)` at the first line of `read_last_line`. – falsetru Nov 26 '20 at 05:38
  • i am sorry, the `b''` fixed the error, but do you know why appending a line at the bottom of the file using text editor will result in null ? – Potato is my name Nov 26 '20 at 05:39
  • @Potatoismyname, What do you mean by `null`? empty string? – falsetru Nov 26 '20 at 05:39
  • yes, i append the line in sample.txt and `f.read(1) == b''` returns true. Why does it not get the last line that i appended? but using program to append to sample.txt is does not have such issue – Potato is my name Nov 26 '20 at 05:42
  • @Potatoismyname, `f.read(1) == b''` means there's nothing to read more. IOW, There's only single line (with or without newline) or there's no line at all (0 byte file) – falsetru Nov 26 '20 at 05:44
  • thank you so much for explaining and answering me!! – Potato is my name Nov 26 '20 at 05:44
  • @Potatoismyname, You're welcome. Happy programming! – falsetru Nov 26 '20 at 05:45