0

I have been trying to fix this for a number of days now without any success. I know I have created another post related to this issue but not sure if I should have continued with the other post rather than creating a new one as I am still quite new to how SO works so apologies if I have gone about this the wrong way.

Objective
Read a text file from disk, output the contents to a Textbox so I can then extract the last 3 lines from it. This is the only way I can think of doing this.

The text file is continuously been updated by another running program but I can still read it even though it is in use but cannot write to it.

I am probing this file through a Timer which ticks every 1 second in order to get the latest information.

Now to the issue...
I have noticed that after some time my app becomes sluggish which is noticeable when I try to move it across the screen or resize it and the CPU usage starts to creep up to over 33%

My Thought Process
As this reading the file is a continuous one, I was thinking that I could move it onto a BackgroundWorker which from my understanding would put it on a different thread and take some load off the main GUI.

Am I barking up the wrong tree on this one?

I am reaching out to more advanced users before I start to get all the text books out on learning how to use the BackgroundWorker.

Here is the code I am using to Read the txt file and output it to a text Box. I have not included the code for extracting the last 3 lines because I don't think that part is causing the issue.

I think the issue is because I am constantly probing the source files every second with a timer but not 100% sure to be honest.

 Dim strLogFilePath As String
    strLogFilePath = "C:\DSD\data.txt"

    Dim LogFileStream As FileStream
    Dim LogFileReader As StreamReader

    'Open file for reading
    LogFileStream = New FileStream(strLogFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)

    LogFileReader = New StreamReader(LogFileStream)

    'populate text box with the contents of the txt file
    Dim strRowText As String
    strRowText = LogFileReader.ReadToEnd()

    TextBox1.text = strRowText

    'Clean Up
    LogFileReader.Close()
    LogFileStream.Close()

    LogFileReader.Dispose()
    LogFileStream.Dispose()
Ian Barber
  • 133
  • 8
  • If you don't need to see on your screen the whole content of the file then do not put it inside a textbox. This op, particularly if the file is big, could by itself a source of performance problems. – Steve Aug 16 '20 at 14:33
  • I think it could be the source of my issue and NO I dont need to see the whole content, I am only interested in the last 3 lines at any given time. How would you store the data if not in a text box – Ian Barber Aug 16 '20 at 14:40
  • 1
    Actually you have your storage in the variable _strRowText_, but you can also use ReadAllLines or ReadLines and keep the last three lines loaded – Steve Aug 16 '20 at 14:51
  • I appreciate that you might want to solve this yourself, but just wanted to let you know this wheel already exists - an app called `mTAIL` – Caius Jard Aug 16 '20 at 15:01
  • 1
    It is a normal mishap as written, TextBox quickly gets very inefficient. A simple way to avoid the problem is to just not close the file, you'll only get the added text. – Hans Passant Aug 16 '20 at 15:02
  • Probably find that a background worker would help.... for a bit, but eventually you will find yourself in the same issue. How big might this text file become? you say you only need the last 3 lines of it, what do you need the last three lines for? – Hursey Aug 16 '20 at 21:11
  • So far after about 1 hour running, the text file from which I am reading from is sat at 400k in size with 18,664 lines. The last 3 lines are sent to a Label on the Form – Ian Barber Aug 17 '20 at 15:54

1 Answers1

1

Firstly, you should use the Using keyword instead of manually disposing objects, because that way you are guaranteed that the object will get disposed, even if an unexpected exception occurs, for example:

' You can initialize variables in one line
Dim strLogFilePath As String = "C:\DSD\data.txt"

Using LogFileStream As New FileStream(strLogFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
    ' Everything goes in here
End Using

You don't need the reader for my solution. The reading will be done manually.

Next, you need to read the last n lines (in your case, 3) of the stream. Reading the entire file when you're only interested in a few lines at the end is inefficient. Instead, you can start reading from the end until you've reached three (or any number of) line seprators (based on this answer):

Function ReadLastLines(
    NumberOfLines As Integer, Encoding As System.Text.Encoding, FS As FileStream,
    Optional LineSeparator As String = vbCrLf
) As String
    Dim NewLineSize As Integer = Encoding.GetByteCount(LineSeparator)
    Dim NewLineCount As Integer = 0
    Dim EndPosition As Long = Convert.ToInt64(FS.Length / NewLineSize)
    Dim NewLineBytes As Byte() = Encoding.GetBytes(LineSeparator)
    Dim Buffer As Byte() = Encoding.GetBytes(LineSeparator)

    For Position As Long = NewLineSize To EndPosition Step NewLineSize
        FS.Seek(-Position, SeekOrigin.End)
        FS.Read(Buffer, 0, Buffer.Length)

        If Encoding.GetString(Buffer) = LineSeparator Then
            NewLineCount += 1

            If NewLineCount = NumberOfLines Then
                Dim ReturnBuffer(CInt(FS.Length - FS.Position)) As Byte
                FS.Read(ReturnBuffer, 0, ReturnBuffer.Length)
                Return Encoding.GetString(ReturnBuffer)
            End If
        End If
    Next

    ' Handle case where number of lines in file is less than NumberOfLines
    FS.Seek(0, SeekOrigin.Begin)
    Buffer = New Byte(CInt(FS.Length)) {}
    FS.Read(Buffer, 0, Buffer.Length)
    Return Encoding.GetString(Buffer)
End Function

Usage:

Using LogFileStream As New FileStream(strLogFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
    ' Depending on system, you may need to supply an argument for the LineSeparator param
    Dim LastThreeLines As String = ReadLastLines(3, System.Text.Encoding.UTF8, LogFileStream)
    ' Do something with the last three lines
    MsgBox(LastThreeLines)
End Using

Note that I haven't tested this code, and I'm sure it can be improved. It may also not work for all encodings, but it sounds like it should be better than your current solution, and that it will work in your situation.

Edit: Also, to answer your question, IO operations should usually be performed asynchronously to avoid blocking the UI. You can do this using tasks or a BackgroundWorker. It probably won't make it faster, but it will make your application more responsive. It's best to indicate that something is loading before the task begins.

If you know when your file is being written to, you can set a flag to start reading, and then unset it when the last lines have been read. If it hasn't changed, there's no reason to keep reading it over and over.

Audiopolis
  • 530
  • 5
  • 24