Serial port data precise time stamp

Question

I am working on a project which requires precise time(ms) for each data entry I read from a serial port connected to an encoder (US digital S5 Optical Shaft Encoder with a QSB).

I installed the encoder on a small cart where I use it to count the speed of the cart.

Here is what I did so far:

1. connect to the serial port and write command to QSB to tell the encoder to stream data. commands available here:

   www.usdigital.com/assets/general/QSB%20Commands%20List_1.pdf www.usdigital.com/assets/general/QSB%20Applications%20Examples.pdf

2. Use readline() to read received data.

3. put all lines of data into one StringBuilder and output it to a file.

I am able to get data entries in between 1ms when I set the output value threshold and interval rate to as fast as possible. Here is what I got:

----time stamp(h/m/s/ms)-------value

data with correct time stamp: https://www.dropbox.com/s/pvo1dz56my4o99y/Capture1.JPG

However, there are abrupt "jumps", roughly 200ms when data is continuous (I am rolling the cart in a constant speed)

data with incorrect time stamp: https://www.dropbox.com/s/sz3sxwv4qwsb2cn/Capture2.JPG

Here is my code:

private void buttonOpenEncoderPort_Click(object sender, EventArgs e)
    {
        serialPortEncoder.Write("S0E\r\n");//start streaming data
        System.Threading.Thread.Sleep(500);
        serialPortEncoder.Write("W0B0\r\n");//set threshold to 0 so the encoder will stream data a the interval I set.
        System.Threading.Thread.Sleep(500);
        serialPortEncoder.Write("W0C0000\r\n");//set output interval to 0 so it will stream as fast as possible
        System.Threading.Thread.Sleep(1500);
        backgroundWorkerEncoder.RunWorkerAsync();}
        //I am using a background worker to pull data out.


 private void backgroundWorkerEncoder_DoWork(object sender, DoWorkEventArgs e)
    {
        while (serialPortEncoder.IsOpen)
        {
            if (serialPortEncoder.BytesToRead != 0)
            {
                try
                {
                    String s = serialPortEncoder.ReadLine();//read from encoder
                    LazerBucket.Add(getCurrentTimeWithMS(timeEncoder) + "-----" + s + "\r\n");//put one line of data with time stamp in a List<String>
                    richTextBoxEncoderData.BeginInvoke(new MethodInvoker(delegate()
                    {
                        richTextBoxEncoderData.Text = s; })); //update UI

                }
                catch (Exception ex) { MessageBox.Show(ex.ToString()); }                   
            }

        }
    }

private String getCurrentTimeWithMS(DateTime d)//to get time
    {
        StringBuilder s = new StringBuilder();
        d = DateTime.Now;
        int hour = d.Hour;
        int minute = d.Minute;
        int second = d.Second;
        int ms = d.Millisecond;
        s.Append("  ----" + hour.ToString() + ":" + minute.ToString() + ":" + second.ToString() + ":" + ms.ToString());
        return s.ToString();
    }

I would appericiate it if someone could find the cause of the time jump. 200ms is too much to be ignored.

EDIT: 

As suggested, I tried Stopwatch but still there are 200ms delay. But when I print out time stamps and BytesToRead together, I found that data in the buffer is decreasing as readLine() is being executed. Eventually BytesToRead will drop to single digit and that's where the delay happens. I am looking for better solutions on how to implement threads. And also explanations for the delay. Maybe I am reading to fast so the buffer can't keep up with me?

EDIT:

problem solved. see my answer below. Thanks for replying though. Stopwatch really helps. Now I am trying to work out whether event driven or polling is better.

Answer 1

After some endless researching on the web, I found the cause of the delay. Device Manager--->Port---->advance----> change latency to 1ms will solve the problem. I am now polling data using a separate thread. It works very well.

Answer 2

Are you using C# 4.5? If so, I highly recommend using async/await over BackgroundWorker.

Also, DateTime isn't really accurate for real-time applications. I would recommend DateTime strictly as a start time and then using Stopwatch in System.Diagnostics to get the elapsed time since the start time.

private void backgroundWorkerEncoder_DoWork(object sender, DoWorkEventArgs e)
{
  var startTime = DateTime.Now;
  var stopwatch = Stopwatch.StartNew();

  while (serialPort.IsOpen && !backgroundWorker.CancellationPending)
  {
    if (serialPort.BytesToRead > 0)
    {
      try
      {
        var line = serialPort.ReadLine();
        var timestamp = (startTime + stopwatch.Elapsed);

        var lineString = string.Format("{0}  ----{1}", 
                                       line,
                                       timestamp.ToString("HH:mm:ss:fff"));

        // Handle formatted line string here.
      }
      catch (Exception ex)
      {
        // Handle exception here.
      }
    }
  }

As for the 200 ms discrepancy, it could be a variety of things. Perhaps the BackgroundWorker is on a lower priority and doesn't get as much CPU time as you hoped. Could also be something on the I/O side of either SerialPort or the actual serial device itself.

Answer 3

When you want precise measurements you should not use DateTime.Now, try stopwatch instead. As detailed here and here, DateTime is accurate but not precise to the millisecond. If you need precision and accuracy, save DateTime.Now when you start measuring and get the offset from the stopwatch.

While 200ms seems like a long delay - even for DateTime - the stopwatch might indeed solve your Problem.

Answer 4

To me it seems that the OS is [in your way].

I suggest the following.

1. Read the data from the port either in a seperate proces (or service) or a separate thread with priority above normal

2. Store the raw(!) data in a queue with the accurate timestamp for later processing. This "task" should be as light as possible to avoid the GC or scheduler to kick in and stall it for even the smallest amount of time. No string concats or formats for example. Those ops cost time and put stress on memory.

3. Process that data in a seperate thread or process. If that one get's hold up for a some time there's no real harm done as the timestamps are accurate.

In short; decouple reading from processing.

Imo stock-versions of Windows are too much IO-bound (it loves and hugs the concept of swapping) to be for RT processes. Using a diff OS, on a diff box, for reading and sending it to a Winbox for further processing could perhaps be a option to consider too (last resort?)