BackgroundWorker & Timer, reading only new lines of a log file?

Question

My application writes a log file (currently using log4net). I'd like to setup a timer and a background worker to read the log file and print its content into some control in my form, while it's being written.

I can't use the FileSystemWatcher class because seems broken: sometimes the event "changed" fires, sometimes do not. And it has an extremely low "pooling rate".

So I created a Timer and a FileSystemWatcher. On the "tick" event of the timer, the background worker does its job.

The question is: how to read only the lines that are added since the last check of the worker?

public LogForm()
{
    InitializeComponent();
    logWatcherTimer.Start();
}

private void logWatcherTimer_Tick(object sender, EventArgs e)
{
    FileInfo log = new FileInfo(@"C:\log.txt");
    if(!logWorker.IsBusy) logWorker.RunWorkerAsync(log);
}

private void logWorker_DoWork(object sender, DoWorkEventArgs e)
{
    // Read only new lines since last check.
    FileInfo log = (FileInfo) e.Argument;

   // Here is the main question!
}

EDIT: Code Solution (maybe there is a more elegant way?):

private void logWatherWorker_DoWork(object sender, DoWorkEventArgs e)
{
    // retval
    string newLines = string.Empty;
    FileInfo log = (FileInfo) e.Argument;

    // Just skip if log file hasn't changed
    if (lastLogLength == log.Length) return;

    using (StreamReader stream = new StreamReader(log.FullName))
    {
        // Set the position to the last log size and read
        // all the content added
        stream.BaseStream.Position = lastLogLength;
        newLines = stream.ReadToEnd();
    }

    // Keep track of the previuos log length
    lastLogLength = log.Length;

    // Assign the result back to the worker, to be
    // consumed by the form
    e.Result = newLines;
}

Answer 1

Check and store the file size each time you read the log, then start your text reader (or whatever you're using) at that location the next time you read.

Answer 2

You could keep track of the index of the last character read from the stream, and subsequently seek to that position.

Edit: see http://dotnetperls.com/seek for examples.

Answer 3

If all you want is to view you log file on a form as it is being written, why not do something simple like write your own Appender that is backed by a TextBox, RichTextBox, or whatever.

Here are some links that I found just doing a quick Google search for "log4net textbox appender":

http://www.nimblecoder.com/blog/archive/2009/01/30/using-a-delegate-and-custom-appender-with-log4net-to-display.aspx (This one looks pretty cool because it allows you to specify a delegate to execute on every log message, so you would not even be tied to a TextBox. You could write different delegates depending on where you wanted your log output to go).

http://www.l4ndash.com/Log4NetMailArchive%2Ftabid%2F70%2Fforumid%2F1%2Fpostid%2F15133%2Fview%2Ftopic%2FDefault.aspx

http://weblogs.asp.net/psteele/archive/2010/01/25/live-capture-of-log4net-logging.aspx

http://www.l4ndash.com/Log4NetMailArchive%2Ftabid%2F70%2Fforumid%2F1%2Fpostid%2F14923%2Fview%2Ftopic%2FDefault.aspx (This one is an Appender that raises an event for every message that is logged).

http://markmail.org/message/ma62bdjpmab3cn7y (relatively recent - posted in 2008 - uses RichTextBox to generated ColoredConsoleAppender-style output)

http://www.claassen.net/geek/blog/2005/06/log4net-scrollingtextbox.html (This one uses the MemoryAppender to capture the log messages and then writes those messages to a TextBox)

http://code.google.com/p/devdefined-tools/source/browse/trunk/projects/common/DevDefined.Common/Appenders/TextBoxAppender.cs?r=90

I have not tried any of these, so I can't vouch for their quality. But, I think that the approach of using a custom Appender backed by a TextBox seems like a much better approach than trying to watch the log file, read it, and then put the messages in a TextBox.

Some common themes that I noticed while looking briefly over these Appenders:

1. When you write to the TextBox from the Appender, you might need to use BeginInvoke.

2. One tricky part seems to be telling the Appender which TextBox to write to. In most cases, the Appender is configured via the config file and then the TextBox is added to the Appender programmatically AFTER the logging system has been initialized (I think you have to either retrieve at least one logger or log at least one message to force all of the lazy initialization to happen).

3. Be careful about constantly adding lines to the TextBox. You could use up a lot of memory, cause performance issues, or exceed the limit on the TextBox (if there is one). Several of these Appenders include code that removes "old" lines from the TextBox periodically.