Reading live data using PyQtGraph without appending the data

Question

I am trying to plot live data using PyQtgraph, which i started to learn. I read a lot of posts here, such as:

Fast plotting using pyqtgraph

and searched for the docs in the internet. The problem that i`m having is that the plot is very slow while drawing the data. This is because, when the data comes from a serial port, I append it to a list, and then I take it from that list and draw it. So, the plot keeps drawing even after I shut down the device where the data is coming from.

This is my code:

class LivePlot(QtGui.QApplication):
  def __init__(self, *args, **kwargs):
    super(MyApplication, self).__init__(*args, **kwargs)
    self.t = QTime()
    self.t.start()

    self.data = deque()
    self.plot = self.win.addPlot(title='Timed data') 
    self.curve = self.plot.plot()

    print "Opening port"
    self.raw=serial.Serial("com4",115200)
    print "Port is open"

    self.tmr = QTimer()
    self.tmr.timeout.connect(self.update)
    self.tmr.start(100)

    self.cnt = 0

  def update(self):
    line = self.raw.read()
    ardString = map(ord, line)
    for number in ardString:
        numb = float(number/77.57)
        self.cnt += 1
        x = self.cnt/2

        self.data.append({'x': x , 'y': numb})  
        x = [item['x'] for item in self.data]
        y = [item['y'] for item in self.data]
        self.curve.setData(x=x, y=y)

So, how can I draw the data without append it to a list? I am new with this library so i am confused. Hope you can help me.

---- EDIT ----

I've tried this modification in the code:

class LivePlot(QtGui.QApplication):
  def __init__(self, *args, **kwargs):
    super(MyApplication, self).__init__(*args, **kwargs)
   self.cnt = 0
 #Added this new lists
   self.xData = []
   self.yData = []

  def update(self):
    line = self.raw.read()
    ardString = map(ord, line)
    for number in ardString:
        numb = float(number/77.57)
        self.cnt += 1
        x = self.cnt/2

        self.data.append({'x': x , 'y': numb}) 
        self.xData.append(x)
        self.yData.append(numb) 

    self.curve.setData(x=self.xData, y=self.yData)

But i am having the same problem.

Answer 1

As mentioned in my comments, setData() repaints the whole series. It's very inefficient if you're just modifying / adding a small portion at a time.

My solution is to keep creating and adding new PlotDataItems (or BarGraphItems, ...) to the series, and to set the PlotWidget to repaint only the added / removed item.

The default setting of QGraphicsView's viewportUpdateMode is QGraphicsView::MinimalViewportUpdate but the QGraphicsScene spends a lot of time trying to determine the minimal update. To disable this set the viewportUpdateMode() of the PlotWidget to QGraphicsView::BoundingRectViewportUpdate .

plot = pg.PlotWidget()
plot.setViewportUpdateMode(QGraphicsView.BoundingRectViewportUpdate)

Turning off item indexing is advantageous for dynamic scenes such as realtime charts. By default a BSP tree is used for indexing. Adding and removing items, as well as all QGraphicsScene's location algorithms, have logarithmic time complexity. With no indexing, adding and removing items runs in constant time, item lookup in linear time. Because the tree depth is being optimized as items are added and removed, in scenes with many items and in dynamic scenes, maintaining the index may outweight the speed advantages of a logarithmic lookup.

For dynamic scenes, or scenes with many animated items, the index bookkeeping can outweight the fast lookup speeds.

Qt5 QGraphicsScene::itemIndexMethod

plot.scene().setItemIndexMethod(QGraphicsScene.NoIndex)

Alternatively you can calculate the optimal tree depth beforehand if you know the constraints of the plot (max number of items) and set it fixed:

# scene - total length of the scene in your units (seconds, ...)
# segment - minimal length of a single segment

depth = math.ceil(math.log(scene / segment, 2) + 1)
plot.scene().setBspTreeDepth(depth)

Qt5 QGraphicsScene::bspTreeDepth

Setting the QGraphicsView.DontAdjustForAntialiasing optimization flag and turning off antialiasing altogether also helps. Setting this flag speeds up the painting / repainting.

plot.setAntialiasing(False)
plot.setOptimizationFlag(QGraphicsView.DontAdjustForAntialiasing)

After each new added item, explicitly processing all pending Qt events might also help:

QtGui.QApplication.processEvents()

Another improvement can be achieved by disabling widget updates and repainting except on selected occasions, such as when a new item is added, or when the window is resized.

The reason for this is that if a user clicks away from a window, paint() is being called on PlotWidgets by the window, which triggers repainting of all PlotDataItems. The same when the user clicks back into the window. Most of the time it's completely unnecessary, and freezes the application temporarily if you have a lot of PlotDataItems in the scene.

However method setUpdatesEnabled() cannot be used, because setUpdatesEnabled(True) automatically calls update() on the whole widget which triggers paint() on all items, which is exactly what we want to avoid.

isShown = False

def setUpdatesDisabled(self, widget, value):
    try:
        widget.setAttribute(Qt.WA_UpdatesDisabled, value)
        widget.setAttribute(Qt.WA_ForceUpdatesDisabled, value)
    except AttributeError:
        pass
    for child in widget.children():
        self.setUpdatesDisabled(child, value)

def myWorkMethod(self):
    self.setUpdatesDisabled(self.plot, False)

    ...

    QtGui.QApplication.processEvents()
    self.setUpdatesDisabled(self.plot, True)

def changeEvent(self, event):
    if event.type() == QEvent.ActivationChange:
        if self.isShown:
            self.setUpdatesDisabled(self.plot, False)
        else:
            self.isShown = True

def resizeEvent(self, event):
    self.plot.setUpdatesEnabled(True)
  
    

Probably an even faster, but less flexible solution would be to simply draw it as an QImage and don't use PyQtGraph at all.

Answer 2

For each number you read from the device you will recreate the x and y arrays from scratch. That means, that if you have 1000 data points you will walk a 1000 times through a list of 1000 elements, which amounts to a million steps. As you can imagine, this doesn't scale well. Your execution time grows with a factor of n^2 where n is the number of points you measure. Google 'computational complexity' or 'big O notation' and read about it.

Appending to a Python list (or deque) is fast, the time it takes does not depend on the number of elements in the list (note that this is not True for appending to a Numpy array). If you create self.xData and self.yData lists and then append your data to those list, you can get rid of the inner loops:

def update(self):
    line = self.raw.read()
    ardString = map(ord, line)
    for number in ardString:
        numb = float(number/77.57)
        self.cnt += 1
        x = self.cnt/2

        # Appending goes in constant time 
        self.data.append({'x': x , 'y': numb})  
        self.xData.append(x)    # self.xData should be a list or deque
        self.yData.append(numb) # self.yData should be a list or deque

    # Drawing the plot is now only performed once per update
    self.curve.setData(x=self.xData, y=self.yData)

Also, redrawing the plot, which happens in setData, is an expensive operation and you do this after reading each data point. A better solution would be to simply update the plot once per update. See the example above. This is still ten times a second (at least, if your plotting can keep up with the data).

As a last remark, communication with hardware is a tricky subject, your first attempt will undoubtedly be unstable and full of bugs. If you are doing this as a hobby project then it's a great opportunity to learn. If you are doing this in a professional setting then try if you can find a mentor.