Progressively append and plot new data to a matplotlib line

Question

Question

What is the way to append data to an existing matplotlib line and plot only the added portion of the line without redrawing the whole line?

Comments

Following is a simple code that plots the redraw time vs. the number of times we append a portion of data to the line.

You see the redraw time increases nearly linearly with the total size of data in the line. This points to the fact that the whole line is redrawn. I'm looking for a way to plot only a new portion of the line. In this case, the redraw time is expected to be nearly constant for the code below.

import matplotlib.pyplot as plt
import numpy as np
import time

# User input
N_chunk = 10000
N_iter = 100

# Prepare data
xx = list(range(N_chunk))
yy = np.random.rand(N_chunk).tolist()

# Prepare plot
fig, ax = plt.subplots()
ax.set_xlim([0,N_chunk])  # observe only the first chunk
line, = ax.plot(xx,yy,'-o')
fig.show()

# Appending data and redraw
dts = []
for i in range(N_iter):
    t0 = time.time()
    xs = xx[-1]+1
    xx.extend(list(range(xs,xs+N_chunk)))
    yy.extend(np.random.rand(N_chunk).tolist())
    line.set_data(xx,yy)
    fig.canvas.draw()
    dt = time.time() - t0
    dts.append(dt)
    plt.pause(1e-10)
plt.close()

# Plot the time spent for every redraw
plt.plot(list(range(N_iter)), dts, '-o')
plt.xlabel('Number of times a portion is added')
plt.ylabel('Redraw time [sec]')
plt.grid()
plt.show()

have you checked this https://stackoverflow.com/a/10944967/6660638 — Epsi95, May 08 '20 at 16:23
I checked it out, thank you. They suggested the similar way of my code: the line is redrawn in whole. Btw, using np.append for big arrays increases the time of each redraw by up to 10 times. — pch, May 08 '20 at 16:37

Sameeresque · Answer 1 · 2020-05-10T18:43:43.197

Here's a modified version of your code that makes use of matplotlib's interactive mode.

import matplotlib.pyplot as plt
import numpy as np
import time

# User input
N_chunk = 10000
N_iter = 100

# Prepare data
xx = np.arange(N_chunk)
yy = np.random.rand(N_chunk)

# Prepare plot
fig, ax = plt.subplots()
#ax.set_xlim([0,N_chunk])  # observe only the first chunk
line = ax.plot(xx,yy,'-o')
plt.ion()   # set interactive mode
fig.show()

# Appending data
dts = []
for i in range(N_iter):
    t0 = time.time()
    xs = xx[-1]+1
    xx=np.arange(xs,xs+N_chunk)
    yy=np.random.rand(N_chunk)
    line=ax.plot(xx,yy,'-o')
    fig.canvas.draw()
    dt = time.time() - t0
    dts.append(dt)
    plt.pause(1e-10)
plt.close()

# Plot the time spent for every redraw
plt.plot(range(N_iter), dts, '-o')
plt.xlabel('Number of times a portion is added')
plt.ylabel('Redraw time [sec]')
plt.grid()
plt.show()

With ax.set_xlim uncommented the redraw times are:

On the other hand, with ax.set_xlim commented:

Clearly, calling fig.canvas.draw() redraws everything. In your case, by commenting ax.set_xlim([0,N_chunk]) you are redrawing things like the axes boundaries, tick labels, etc. You want to explore blitting as discussed in this SO to avoid redrawing axes objects.

First, you changed the code: you add more lines instead of extending the original line. Second, you seem to confuse the commented and the uncommented cases. Its obvious that if you show a bounded portion of data, the rendering takes less time. — pch, May 12 '20 at 20:59
Thanks for the link. *Blitting* helps well when you have many artists in the plot, while you need to update only few of them. In my case the only artist presents. Anyway I tried *blitting* with your approach (add separate line for every new portion of data). It works fine along with update of a background every time in a loop (finally, ~0.03sec per redraw). Although you got multiple lines that are not connected in-between. — pch, May 12 '20 at 21:24
The point of my code is that it does away with extending data. It adds new data while also presenting existing data. Your code, on the other hand, extends the data and plots all the data. — Sameeresque, May 12 '20 at 21:26

Progressively append and plot new data to a matplotlib line

Question

Comments

1 Answers1