1

Hi all PyQtGraph users,

I am trying to use Matplotlibs jet colors in the pg.ScatterPlotItem for real-time graph update using data from a datalogger. I convert jet colors to RGBA code in the following few lines:

z=np.random.randint(0,10000,10000) #random z value
norm = mpl.colors.Normalize(vmin=min(z), vmax=max(z))
m = cm.ScalarMappable(norm=norm, cmap=cm.jet)
colors =  m.to_rgba(z, bytes=True)

I have noticed that the speed in the scatterplot update can be an issue if the number of points is 10k or higher. For instance, on my laptop I get an update speed of 0.16 fps for 10k points if I run example code from the ScatterplotTestSpeed.py. However, if I append lists as tuples I can triple the update speed to 0.45 fps:

colors_=[]
for i in colors:
  colors_.append(pg.mkBrush(tuple(i)))

This is a small trick I discovered by simple try and see method, but I was wondering if there are more of such tricks to speed up the fps number?!

Here is my entire code to test the update speed. It is heavily based on the ScatterPlotSpeedTest.py form the pyqtgraphs example library. Any help is appreciated :-)

#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
For testing rapid updates of ScatterPlotItem under various conditions.

(Scatter plots are still rather slow to draw; expect about 20fps)
"""



## Add path to library (just for examples; you do not need this)
import initExample

import matplotlib as mpl
import matplotlib.cm as cm
from pyqtgraph.Qt import QtGui, QtCore, USE_PYSIDE
import numpy as np
import pyqtgraph as pg
from pyqtgraph.ptime import time
#QtGui.QApplication.setGraphicsSystem('raster')
app = QtGui.QApplication([])
#mw = QtGui.QMainWindow()
#mw.resize(800,800)
if USE_PYSIDE:
    from ScatterPlotSpeedTestTemplate_pyside import Ui_Form
else:
    from ScatterPlotSpeedTestTemplate_pyqt import Ui_Form

win = QtGui.QWidget()
win.setWindowTitle('pyqtgraph example: ScatterPlotSpeedTest')
ui = Ui_Form()
ui.setupUi(win)
win.show()

p = ui.plot
p.setRange(xRange=[-500, 500], yRange=[-500, 500])

data = np.random.normal(size=(50,10000), scale=100)
sizeArray = (np.random.random(500) * 20.).astype(int)
ptr = 0
lastTime = time()
fps = None
def update():
    global curve, data, ptr, p, lastTime, fps
    p.clear()
    if ui.randCheck.isChecked():
        size = sizeArray
    else:
        size = ui.sizeSpin.value()


    z=np.random.randint(0,10000,10000)
    norm = mpl.colors.Normalize(vmin=min(z), vmax=max(z))
    m = cm.ScalarMappable(norm=norm, cmap=cm.jet)
    colors =  m.to_rgba(z, bytes=True)

    colors_=[]
    for i in colors:
      colors_.append(pg.mkBrush(tuple(i)))
      #colors.append(pg.intColor(np.random.randint(0,255), 100))

    curve = pg.ScatterPlotItem(x=data[ptr%50], y=data[(ptr+1)%50],
                   pen='w', brush=colors_, size=size,
                   pxMode=ui.pixelModeCheck.isChecked())


    '''
    curve = pg.ScatterPlotItem(pen='w', size=size, pxMode=ui.pixelModeCheck.isChecked())
    spots3=[]

    for i,j,k in zip(data[ptr%50],data[(ptr+1)%50],colors):
      spots3.append({'pos': (i, j), 'brush':pg.mkBrush(tuple(k))})

    curve.addPoints(spots3)
    '''



    p.addItem(curve)
    ptr += 1
    now = time()
    dt = now - lastTime
    lastTime = now
    if fps is None:
        fps = 1.0/dt
    else:
        s = np.clip(dt*3., 0, 1)
        fps = fps * (1-s) + (1.0/dt) * s
    p.setTitle('%0.2f fps' % fps)
    p.repaint()
    #app.processEvents()  ## force complete redraw for every plot
timer = QtCore.QTimer()
timer.timeout.connect(update)
timer.start(0)



## Start Qt event loop unless running in interactive mode.
if __name__ == '__main__':
    import sys
    if (sys.flags.interactive != 1) or not hasattr(QtCore, 'PYQT_VERSION'):
        QtGui.QApplication.instance().exec_()
  • Why exactly would you need 10000 random colors when plotting data from a logger? If you plot without this color-play-arounds you'll gain more then a factor of ten in framerate. Since I cannot run your code as it's importing stuff I don't have, I wrote another piece of code which results in 11 fps for 10000 points in ScatterPlot with uniform colors; 1.0 fps with 10000 random colors. – ImportanceOfBeingErnest Nov 12 '16 at 20:06
  • Btw. `ScatterPlot` might not be the best option when it comes to speed. For example, in [this post](http://stackoverflow.com/a/40139416/4124317) I answered the other day, we see that you can reach 250 fps with an `ImageItem` of 100 x 100 = 10000 pixels. – ImportanceOfBeingErnest Nov 12 '16 at 20:23
  • It is not 10k random colors but jet coloring due to the value of z. Every time I get a new z-value from my datalogger I need to re-color all existing points already logged. One solution is to set jet colors at fixed z-values but then the coloring will not be fully dynamic. –  Nov 12 '16 at 22:58
  • In order to run the code you need to run it from the example library in your PyQtGraph folder, then all the required ScatterPlotSpeedTest templates will be loaded correctly (3 of them I think). –  Nov 12 '16 at 23:05
  • Anyways, up to now this is a purely theoretical example case, since noone would actually be able to gain much information from a scatter plot with 10k samples. So in order to gain speed it might be good to think more in the direction of your usage case, where (as you said) values are not random any more. With non-random values, some more tricks might be possible. – ImportanceOfBeingErnest Nov 13 '16 at 11:18

1 Answers1

1

I think that I have found a solution to my problem. It seems that ScatterPlotItem is simply too slow to continuously redraw large number of points (>500) with different colors specified by 'brush'. Instead, I am just using plotItem and 'symbolBrush' to color incoming data points with Jet colors (that will give me 2.5D plot I was initially searching for). The speed of the plot update is almost instant no matter how many data points are added, 100 points are recolored just as fast as 10000 points (from the user point of view).