2

I have a windows form application that will have a primary task of reading through each line in a large text file and writing that line out to another file after performing some search and replace on that line. Since the files are very large >20 GB the process is expected to take a long time to complete.

The process of reading, modifying, and writing each line of the large text files will be time consuming so I would like to notify the user of the application of the progress during the process. It could be nothing more than counting the number of lines in the file and then dividing the current line number to get a percentage of work completed.

I know that this is where events and delegates come in but I have to admit I remain confused, even after reading many articles, on how to make use of this pattern. Any help in showing me how to add events to this app would be helpful.

Below is an example of what I am attempting to do. Essential replacing all values of t with x in the file.

WinForm class

namespace ParseFileTool
{
    public partial class FormMain : Form
    {
      public FormMain()
      {
          InitializeComponent();
      }

      private void btnStartProcessing_Click(object sender, EventArgs e)
      {
        ProcessFile pf = new ProcessFile();
        pf.InputFilePath = "PATH TO INPUT FILE";
        pf.OutputFilePath = "PATH TO OUTPUT FILE";
        pf.StartProcessing();
      }
    }
}

File Processor Class

class ProcessFile
{
    public string InputFilePath { get; set; }
    public string OutputFilePath { get; set; }

    public void StartProcessing()
    {
      using (StreamReader sr = new StreamReader(this.InputFilePath))
      {
        string line;
        while ((line = sr.ReadLine()) != null)
        {
          string newLine = line.Replace("t","x");
          using (FileStream fs = new FileStream(this.OutputFilePath, FileMode.Append, FileAccess.Write))
          using (StreamWriter sw = new StreamWriter(fs))
          {
            sw.WriteLine(newLine);
          }    
        }
      }
    }
}

I also wanted to know if I should be kicking off the instance of the ProcessFile class on a separate thread in order to allow the GUI to update properly or is this not necessary because events are to be used.

Thanks for any assistance. I would sure like to become comfortable with events and subscribing to them. I understand what a delegate is but I remain unsure where to declare it (the winform class or the ProcessFile class) and how to handle passing back data (like the line number processed) to the event handler.

webworm
  • 10,587
  • 33
  • 120
  • 217
  • Might want to look into these - http://stackoverflow.com/a/21419280/903324 http://stackoverflow.com/q/9684112/903324 – Abijeet Patro Jun 28 '15 at 15:37
  • http://stackoverflow.com/a/30679912/2901207 Backgroundworker is what you need! explained right here! You can notify progress, cancel work, etc.. – CularBytes Jun 28 '15 at 15:43
  • http://www.oodesign.com/observer-pattern.html consider using the observer pattern to encapsulate the background progress bar code. This pattern will easily allow you to add more observers like a log or email. – MIKE Jun 28 '15 at 15:59
  • @RageComplex - If I understand the example you pointed to it looks like events are built in? Is that correct? No need to declare delegate or event. Is this true? – webworm Jun 28 '15 at 16:00
  • Just a quick note with async versions of .net events can have async added to the, automatically so you can avoid the cumbersome background worker and use the recommended task library. – Po-ta-toe Jun 28 '15 at 16:51

2 Answers2

1

I like the BackgroundWorker for WinForms, although there's also the newer async/await model you can research as well.

Drop a BackgroundWorker onto your FormMain form and set ReportsProgress = true. Subscribe to the available events, for doing work and reporting progress.

Move anything that doesn't touch the UI into the DoWork event, which runs on a separate thread. Modify your loop so it reports progress at some regular interval.

public void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
  var inputFilePath = "input file path";
  var outputFilePath = "output file Path";

  using (var sr = new StreamReader(inputFilePath))
  {
    int counter = 0;

    string line;
    while ((line = sr.ReadLine()) != null)
    {
      // do all the important stuff

      counter++;

      // report progress after every 100 lines
      // reporting too often can essentially lock the UI by continuously updating it
      if (counter % 100 == 0)
        backgroundWorker1.ReportProgress(counter);
    }
  }
}

The ProgressChanged event runs back on the UI thread, so you can accept whatever the DoWork event passed to it and display that to the user.

private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
  lblProgress.Text = string.Format("Processed {0} lines.", e.ProgressPercentage);
}

You'll probably want to subscribe to the RunWorkerCompleted event too...

void backgroundWorker1_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
  lblProgress.Text = "Job finished!";
}

Start it by calling:

backgroundWorker1.RunWorkerAsync();
Grant Winney
  • 65,241
  • 13
  • 115
  • 165
  • Thanks for the great example! How would I pass in the file path names to the `backgroundWorker1_DoWork()` method? Instead of having them hard coded in the method. – webworm Jun 28 '15 at 16:58
  • 1
    Thanks for the tip on the Tuple. I called the RunWorkerAsync method as such `bgWorker.RunWorkerAsync(Tuple.Create(inputFilePath, outputFilePath));` Then inside the `DoWork` method I cast the `e.Argument` as such `var filePaths = e.Argument as Tuple;` – webworm Jun 28 '15 at 17:14
  • How does the `e.ProgressPercentage` know the percentage complete? It looks like the number of lines was completed was passed in. How does it know how many lines were in the file? Do I need to calculate the percentage and send it in? – webworm Jun 28 '15 at 17:32
0

You need to use background thread. Many options are here, but most simple is:

Task.Run(() => pf.StartProcessing());
ranquild
  • 1,799
  • 1
  • 16
  • 25