Accurate measurement of download speed of a webclient

Question

I am using C# and the WebClient class.

This is the code I am using, inspired by another post here on SO. This code worked well for large files, it accurately displayed the download speed. However, there is now the limitation of downloading individual files, many of which are small, with small being .5-5 MB. This has caused the speed counter to skyrocket, often into the hundreds of thousands of KBps. I'm not sure what else to try to combat this. I added a second progress bar showing individual file downloads which helps improve the image a bit, but the download speed counter should really be fixed. Is there a different class to use that would solve this problem?

The WebClient in this code is disposed of properly elsewhere. private class NetSpeedCounter { private double[] DataPoints;

        private DateTime LastUpdate;
        private int NumCounts = 0;
        private int PrevBytes = 0;

        public double Speed { get; private set; }

        public NetSpeedCounter(WebClient webClient, int maxPoints = 10)
        {
            DataPoints = new double[maxPoints];

            Array.Clear(DataPoints, 0, DataPoints.Length);
            webClient.DownloadProgressChanged += (sender, e) =>
            {
                var msElapsed = DateTime.Now - LastUpdate;

                int curBytes = (int)(e.BytesReceived - PrevBytes);
                PrevBytes = (int)e.BytesReceived;

                double dataPoint = ((double)curBytes) / msElapsed.TotalSeconds;
                DataPoints[NumCounts++ % maxPoints] = dataPoint;

                Speed = DataPoints.Average();
            };
        }

        public void Reset()
        {
            PrevBytes = 0;
            LastUpdate = DateTime.Now;
        }
    }

I download the files with this code, which is started afterwards by a call to DownloadFileAsync. This code just downloads them in a chain, one after another, asynchronously.

This is setting up for starting the download Queue recordQ = new Queue(files);

    progressBar.Value = 0;
    progressBar.Maximum = recordQ.Count;

    UpdateStatusText("Downloading " + recordQ.Count + " files");

    var record = recordQ.Dequeue();
    speedUpdater.Start();
    CheckAndCreate(record.AbsolutePath);

Adding the event handler

    wc.DownloadFileCompleted += (sender, e) =>
    {
    var nr = recordQ.Dequeue();
    CheckAndCreate(nr.AbsolutePath);
    this.Invoke((MethodInvoker)delegate
    {
        UpdateStatusText("Downloading " + recordQ.Count + " files", lblStatusR.Text);
    });
    counter.Reset();
    // download the next one
    wc.DownloadFileAsync(nr.DownloadPath, nr.AbsolutePath);
 }
 counter.Start();
 wc.DownloadFileAsync(record.DownloadPath, record.AbsolutePath);

This last call is what starts everything off.

Answer 1

DateTime.Now is not accurate enough for some scenarios where timespans are recorded frequently(Eric Lippert mentions that they have a precision of 30 ms here), since DateTime.Now will return the previously used DateTime.Now when called quickly in succession. This might result in discrepencies in your speed counter due to inaccurate increases when downloads are finished very quickly. I'd recommend using StopWatch API for that purpose.

EDIT

I have created the following test Winforms application based on your code that works fine for small files. I'm getting a reasonable 200 kbps over my intranet for 5 files that are about 2MB each. Just make sure you're calling the stopwatch classes at the right places. To replicate, create a winforms app, create 3 labels of Id lblSpeed, lblStatus, lblFile and copy\paste the code and rename the URI's below to the files you want to test on.

using System;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace WindowsFormsApplication1
{
  public partial class Form1 : Form
  {
    Queue<Record> recordQ;
    WebClient wc;
    NetSpeedCounter counter;
    //We store downloaded files in C:\TestDir (hardcoded in the Record class below)
    public Form1()
    {
      InitializeComponent();
      recordQ = new Queue<Record>();
      //replace the URI string below. Nothing else to replace.
      //recordQ.Enqueue(new Record(@"URI1", "SQLtraining.exe"));
      //recordQ.Enqueue(new Record(@"URI2", "Project Mgmt.pptx"));

      //first uri to process. Second param is the file name that we store.
      Record record = new Record(@"URI0","Agile.pptx"); // replace the URI

      //Initialize a webclient and download the first record
      using (wc = new WebClient())
      {
        counter = new NetSpeedCounter(wc);
        wc.DownloadFileCompleted += (sender, e) =>
        {
          if (recordQ.Count == 0)
          {
            UpdateStatusText("Done");
            return;
          }
          var nr = recordQ.Dequeue();
          //just create directory. the code uses the same directory
          CheckAndCreate(nr.Directory);
          //need not even use invoke here. Just a plain method call will suffice.
          this.Invoke((MethodInvoker)delegate
          {
              UpdateStatusText("Left to process: " + recordQ.Count + " files");
          });
          counter.Reset();
          counter.Start();
          //continue with rest of records
          wc.DownloadFileAsync(nr.DownloadPath, nr.GetFullPath());
          this.lblFile.Text = nr.DownloadPath.OriginalString;
        };
        //just update speed in UI
        wc.DownloadProgressChanged += wc_DownloadProgressChanged;
        counter.Start();
        //display URI we are downloading
        this.lblFile.Text = record.DownloadPath.OriginalString;
        //start first download
        wc.DownloadFileAsync(record.DownloadPath, record.GetFullPath());
      }

    }

    void wc_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
    {
      this.lblSpeed.Text = counter.Speed.ToString();
    }
    public void UpdateStatusText(string msg)
    {
      this.lblStatus.Text = msg;
    }
    public void CheckAndCreate(string absPath)
    {
      if (!Directory.Exists(absPath))
        Directory.CreateDirectory(absPath);
    }   

  }
  public class NetSpeedCounter
  {
    private int NumCounts = 0;
    private int PrevBytes = 0;
    private Stopwatch stopwatch;
    public double Speed { get; private set; }
    double[] DataPoints;
    public NetSpeedCounter(WebClient webClient, int maxPoints = 10)
    {
      DataPoints = new double[maxPoints];
      stopwatch = new Stopwatch();
      Array.Clear(DataPoints, 0, DataPoints.Length);
      webClient.DownloadProgressChanged += (sender, e) =>
      {
        var msElapsed = DateTime.Now - LastUpdate;

        stopwatch.Stop();
        int curBytes = (int)(e.BytesReceived - PrevBytes);
        PrevBytes = (int)e.BytesReceived;
        //record in kbps
        double dataPoint = (double)curBytes / (stopwatch.ElapsedMilliseconds); 
        DataPoints[NumCounts++ % maxPoints] = dataPoint;
        //protect NumCount from overflow
        if (NumCounts == Int32.MaxValue)
          NumCounts = 0;

        Speed = DataPoints.Average();
        stopwatch.Start();
      };
    }

    public void Start()
    {
      stopwatch.Start();
    }

    public void Reset()
    {
        PrevBytes = 0;            
        stopwatch.Reset();
    }
  }
  public class Record
  {
    public string Directory;
    public string File;
    public Uri DownloadPath;
    public Record(string uriPath, string fileOutputName)
    {
      this.Directory = @"C:\TestDir\";
      this.DownloadPath = new Uri(uriPath);
      this.File = fileOutputName;
    }
    public string GetFullPath()
    {
      return this.Directory + this.File;
    }
  }  

}

Answer 2

I have come up with a different way of doing it.

Instead of adding a data point for each ProgressChanged event, I now increment a counter with the amount of bytes. This counter stays across multiple files, and then I simply take a datapoint using a timer, and get the time in between using a System.Diagnostic.Stopwatch

It works very good, the accuracy is much much better, I would highly recommend doing it this way.

Here is some code

    class NetSpeedCounter
    {
        private Stopwatch watch;
        private long NumCounts = 0;
        private int PrevBytes = 0;
        private double[] DataPoints;

        private long CurrentBytesReceived = 0;

        public double Speed { get; private set; }

        private System.Timers.Timer ticker = new System.Timers.Timer(100);

        public NetSpeedCounter(WebClient webClient, int maxPoints = 5)
        {
            watch = new System.Diagnostics.Stopwatch();

            DataPoints = new double[maxPoints];

            webClient.DownloadProgressChanged += (sender, e) =>
            {
                int curBytes = (int)(e.BytesReceived - PrevBytes);
                if (curBytes < 0)
                    curBytes = (int)e.BytesReceived;

                CurrentBytesReceived += curBytes;

                PrevBytes = (int)e.BytesReceived;
            };

            ticker.Elapsed += (sender, e) =>
                {
                    double dataPoint = (double)CurrentBytesReceived / watch.ElapsedMilliseconds;
                    DataPoints[NumCounts++ % maxPoints] = dataPoint;
                    Speed = DataPoints.Average();
                    CurrentBytesReceived = 0;
                    watch.Restart();
                };
        }

        public void Stop()
        {
            watch.Stop();
            ticker.Stop();
        }

        public void Start()
        {
            watch.Start();
            ticker.Start();
        }

        public void Reset()
        {
            CurrentBytesReceived = 0;
            PrevBytes = 0;
            watch.Restart();
            ticker.Start();
        }
    }