Read Changes on a text file dynamically c#

Question

I have a program that continuously writes its log to a text file. I don't have the source code of it, so I can not modify it in any way and it is also protected with Themida.

I need to read the log file and execute some scripts depending on the content of the file. I can not delete the file because the program that is continuously writing to it has locked the file. So what will be the better way to read the file and only read the new lines of the file? Saving the last line position? Or is there something that will be useful for solving it in C#?

Related to http://stackoverflow.com/questions/13185319/how-to-detect-real-time-change-of-text-files — Michael J. Gray, Oct 07 '13 at 16:48

score 6 · Accepted Answer · edited May 23 '17 at 12:00

Perhaps use the FileSystemWatcher along with opening the file with FileShare (as it is being used by another process). Hans Passant has provided a nice answer for this part here:

var fs = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); 
        using (var sr = new StreamReader(fs)) {
            // etc...
        }

Have a look at this question and the accepted answer which may also help.

Trevor Elliott · Answer 2 · 2013-10-07T16:42:52.417

4

using (var fs = new FileStream("test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete))
using (var reader = new StreamReader(fs))
{
    while (true)
    {
        var line = reader.ReadLine();

        if (!String.IsNullOrWhiteSpace(line))
            Console.WriteLine("Line read: " + line);
    }
}

I tested the above code and it works if you are trying to read one line at a time. The only issue is that if the line is flushed to the file before it is finished being written then you will read the line in multiple parts. As long as the logging system is writing each line all at once it should be okay.

If not then you may want to read into a buffer instead of using ReadLine, so you can parse the buffer yourself by detecting each Environment.NewLine substring.

edited Oct 07 '13 at 16:42

answered Oct 07 '13 at 16:29

Trevor Elliott

11,292
11
63
102

Also a good answer, it waits until the full line is available, but it will lock the file continuously (which isn't necessarily a problem). – Kind Contributor Feb 27 '17 at 23:12
one addition here would be to add bit of Sleep if line == NullOrWhiteSpace. – AAATechGuy Mar 21 '21 at 03:14

score 3 · Answer 3 · answered Oct 07 '13 at 16:44

3

You can just keep calling ReadToEnd() in a tight loop. Even after it reaches the end of the file it'll just return an empty string "". If some more data is written to the file it will pick it up on a subsequent call.

while (true)
{
    string moreData = streamReader.ReadToEnd();
    Thread.Sleep(100);
}

Bear in mind you might read partial lines this way. Also if you are dealing with very large files you will probably need another approach.

answered Oct 07 '13 at 16:44

Weyland Yutani

4,682
1
22
28

This is a great answer, very simple. Mine is only better because the file is not read-locked most of the time. – Kind Contributor Feb 27 '17 at 23:11

score 2 · Answer 4 · answered Oct 07 '13 at 17:02

2

Use the filesystemwatcher to detect changes and get new lines using last read position and seek the file.

http://msdn.microsoft.com/en-us/library/system.io.filestream.seek.aspx

answered Oct 07 '13 at 17:02

Francis

367
1
10

score 1 · Answer 5 · edited May 23 '17 at 12:25

The log file is being "continuously" updated so you really shouldn't use FileSystemWatcher to raise an event each time the file changes. This would be triggering continuously, and you already know it will be very frequently changing.

I'd suggest using a timer event to periodically process the file. Read this SO answer for a good pattern to use System.Threading.Timer¹. Keep a file stream open for reading or reopen each time and Seek to the end position of your last successful read. By "last successful read" I mean that you should encapsulate the reading and validating of a complete log line. Once you've successfully read and validated a log line, then you have a new position for the next Seek.

¹ Note that System.Threading.Timer will execute on a system supplied thread that is kept in business by the ThreadPool. For short tasks this is more desirable that a dedicated thread.

Could you be more detailed please? I have more or less the same problem. I have a big file (60MB) which is written all the time from a machine. Now i should read all the data which is written in there. Could i also delete the file from time to time? — Lukas Hieronimus Adler, Apr 11 '17 at 08:14

score 1 · Answer 6 · edited May 23 '17 at 12:31

1

Use this answer on another post c# continuously read file.

This one is quite efficient, and it checks once per second if the file size has changed. So the file is usually not read-locked as a result.

The other answers are quite valid and simple. A couple of them will read-lock the file continuously, but that's probably not a problem for most.

edited May 23 '17 at 12:31

Community

1
1

answered Feb 27 '17 at 23:09

Kind Contributor

17,547
6
53
70

Read Changes on a text file dynamically c#

6 Answers6

Linked