1

I think this is a fun problem. I'm currently working to read in UDP data from a client whose line by line output looks like this:

X,-3.6958605654872471e+001,Y,-1.6998255581789078e+001,Z,-8.2844374603286042e+000
X,-3.6958587424752295e+001,Y,-1.6998317167303274e+001,Z,-8.2844126284670612e+000
X,-3.6958772663292550e+001,Y,-1.6998425235838699e+001,Z,-8.2842123939323624e+000

The next step is to read in the raw data from the server and plot the data as it comes in live on a 3d scatter plot. I have found relevant answers on Stackoverflow on how to plot data in real time. But these answers are simple because only one point is created for each index in a for loop.

This is the code for my server

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import socket

UDP_IP_ADDRESS = ''
UDP_PORT_NO = 5005

serverSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
serverSock.bind((UDP_IP_ADDRESS, UDP_PORT_NO))

print "Server started from port:", UDP_PORT_NO
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
plt.ion()

while True:
    raw_data, addr = serverSock.recvfrom(4096)
    x,y,z = parse_data(raw_data)              #parse line into 3 distinct values
    ax.scatter(x, y, z)
    plt.pause(0.0001)

As you can see, I'm reading points as fast as they come, which poses the question.

Can python handle receiving one point at a time or should I spin up a thread for every parse_data() call so that all points can be plotted? If I use a thread however, then I fear a race condition may cause the plotted points to be plotted in the wrong order. The solution may require me to use a message queue and mutexes.

What solution would guarantee order of incoming messages and give the lowest latency?

Max
  • 2,072
  • 5
  • 26
  • 42
  • Sorry, can you clarify the issue here? It's that you only have 1 point live on the plot at any one time, or that you can't plot fast enough for the incoming requests? – roganjosh Oct 13 '17 at 20:22
  • It's that I cannot plot fast enough, which to me means I likely need to spin up a separate thread to keep up with the incoming data – Max Oct 13 '17 at 20:24
  • I really doubt that threading will help you speed this up; the GIL ensures that only 1 thread executed at one time. I think that will propagate all the way through mpl. – roganjosh Oct 13 '17 at 20:26
  • Also, how are you rendering the plot? There might be a more suitable library for this, especially if it's in a browser. Interesting question. – roganjosh Oct 13 '17 at 20:27
  • 1
    You're using a UDP socket. You've already lost any ordering guarantees. And are you sure you must see each data point *immediately* upon arrival? Batching the plot updates into one or a few each second would be much more efficient, especially given you'll likely not be able to react to any one point before many new ones arrive. – bnaecker Oct 13 '17 at 21:41
  • I actually do think that threading is a good idea here. This will not solve the problem of ordering though, because that may get wrong on the network level already. But you may plot in the main thread and have another thread collecting the data. Once in a while (as fast as matplotlib allows you) you update the plot with what the other thread as collected so far, possibly plotting a couple of new points at a time. – ImportanceOfBeingErnest Oct 14 '17 at 13:17

0 Answers0