Accurate timer with PyQt

Question

I'm using pyqtgraph to plot a huge number of data that I receive from sensors.

To do so, I made one thread that acquire the data and put in a queue. To plot the data, I check periodically with a timer if the queue is not empty. The problem is that the accuracy of the timer (QTimer) seems to be really bad. I mean when the load is low (sleep for 1000/100 ms) in the measuring thread, the accuracy is pretty good but when the load increase (sleep for 10ms), my update function used to plot data is not called back with the same period.

Here is an example code:

import sys
import time
from queue import Queue
from random import random

import numpy as np
import pyqtgraph as pg
from pyqtgraph.Qt import QtCore, QtWidgets

data_queue = Queue()


class WorkerThread(QtCore.QThread):
    def __init__(self, parent):
        super(WorkerThread, self).__init__(parent=parent)

    def run(self):
        t_init = time.time()
        while True:
            # Generating random data
            values = [(time.time()-t_init, random()) for _ in range(200)]
            data_queue.put(values)
            print("adding data")
            self.msleep(10)


class GraphPlot(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        super(GraphPlot, self).__init__(parent)

        self.mainbox = QtWidgets.QWidget()
        self.setCentralWidget(self.mainbox)
        self.mainbox.setLayout(QtWidgets.QVBoxLayout())

        self.canvas = pg.GraphicsLayoutWidget()
        self.mainbox.layout().addWidget(self.canvas)

        self.analogPlot = self.canvas.addPlot(title='Real-time data')
        self.drawplot = self.analogPlot.plot(pen='r')

        numPoints = 20000
        self.t = np.zeros(numPoints, dtype=int)
        self.x = np.zeros(numPoints, dtype=int)

        self.worker = WorkerThread(self)
        self.worker.start()

        self.timer = pg.QtCore.QTimer()
        self.timer.setTimerType(QtCore.Qt.PreciseTimer)
        self.timer.timeout.connect(self._update)
        self.timer.start(1)

    def _update(self):
        print('start:', time.time())
        size = data_queue.qsize()
        if size > 0:
            for _ in range(size):
                values = data_queue.get()
                for v in values:
                    self.t = np.append(self.t[1:], v[0])
                    self.x = np.append(self.x[1:], v[1])
            self.drawplot.setData(self.t, self.x)        
        print('end:', time.time())


app = QtWidgets.QApplication(sys.argv)
plot = GraphPlot()
plot.show()
sys.exit(app.exec_())

An excerpt of the output:

start: 1572893919.9067862
adding data
end: 1572893919.9217482    <-- 
adding data
start: 1572893919.9586473  <-- there should be 1ms of difference with last 'end'
                               actually, there is around 37ms

I want the timer to be synchronous with the same period whatever the load on the measuring thread. I tried to decrease the priority of the former thread but it did not solve the problem.

Answer 1

QTimer documentation partially answers to your issue:

All timer types may time out later than expected if the system is busy or unable to provide the requested accuracy. In such a case of timeout overrun, Qt will emit timeout() only once, even if multiple timeouts have expired, and then will resume the original interval.

The problem is that after you call _update from the timeout, Qt will need some time to process what happens after self.drawplot.setData(), which basically is computing graphical information and actually painting it on the screen.
You're not getting the 1ms delay because Qt just isn't able to work that fast.

Even if a QTimer can work in another thread ("asynchronously", but be careful about the meaning of this word), it always depend on the thread it is created or resides (a QTimer cannot be started or stopped from a thread different than its one). So, since you've created the timer in the window thread (the Qt main event loop), its timeout accuracy depends on the capacity of that loop to handle all its events, and since lot of events are related to GUI painting (which seems fast to our eyes, but is actually slow as it's very demanding for the CPU), you can easily understand why you'll never get that 1ms interval. And don't forget the fact that even if pyqtgraph is very fast we're still talking about Python.

While reaching a better accuracy for a 1ms QTimer is possible (creating a separate thread for it), you wouldn't get any advantage from it anyway: even with a very fast computer, what you're substantially requesting is to update the screen at 1000Hz, while most graphic hardware is not able to go much faster than 100-200Hz; this means that even if you own a high end system, you wouldn't get more than one update each 4ms.

If you need to update the plot each time new data is available, it is probably better to use signals and slots, which also avoids any unnecessary check on the queue and ensures to update the plot as much as needed:

class WorkerThread(QtCore.QThread):
    newData = QtCore.pyqtSignal(object)
    def __init__(self, parent):
        super(WorkerThread, self).__init__(parent=parent)

    def run(self):
        t_init = time.time()
        while True:
            # Generating random data
            values = [(time.time()-t_init, random()) for _ in range(200)]
            print("adding data")
            self.newData.emit(values)
            self.msleep(10)

class GraphPlot(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        super(GraphPlot, self).__init__(parent)

        self.mainbox = QtWidgets.QWidget()
        self.setCentralWidget(self.mainbox)
        self.mainbox.setLayout(QtWidgets.QVBoxLayout())

        self.canvas = pg.GraphicsLayoutWidget()
        self.mainbox.layout().addWidget(self.canvas)

        self.analogPlot = self.canvas.addPlot(title='Real-time data')
        self.drawplot = self.analogPlot.plot(pen='r')

        numPoints = 20000
        self.t = np.zeros(numPoints, dtype=int)
        self.x = np.zeros(numPoints, dtype=int)

        self.worker = WorkerThread(self)
        self.worker.newData.connect(self.newData)
        self.worker.start()

    def newData(self, data):
        print('start:', time.time())
        for v in data:
            self.t = np.append(self.t[1:], v[0])
            self.x = np.append(self.x[1:], v[1])
        self.drawplot.setData(self.t, self.x)
        print('end:', time.time())

You won't get a 1ms update, but there would be no need for it anyway; also, remember that printing at that rate will always affect the performance in some way.

Finally, there is no advantage in setting PreciseTimer with a 1ms interval, since the timer accuracy is about 1ms on most platforms anyway (as explained at the beginning of the same paragraph in the documentation linked before), and setting the precision is only required for longer intervals (I'd say at least 25-50ms).

There's also an interesting answer about QTimer here, explaining what basically happens whenever you create one and it timeouts.