I have a M x N 2D array: ith row represents that value of N points at time i.
I want to visualize the points [1 row of the array] in the form of a graph where the values get updated after a small interval. Thus the graph shows 1 row at a time, then update the values to next row, so on and so forth.
I want to do this in a jupyter notebook. Looking for reference codes.
I tried following things but no success:
Here's an alternative, possibly simpler solution:
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
m = 100
n = 100
matrix = np.random.normal(0,1,m*n).reshape(m,n)
fig = plt.figure()
ax = fig.add_subplot(111)
plt.ion()
fig.show()
fig.canvas.draw()
for i in range(0,100):
ax.clear()
ax.plot(matrix[i,:])
fig.canvas.draw()
I had been particularly looking for a good answer for the scenario where one thread is pumping data and we want Jupyter notebook to keep updating graph without blocking anything. After looking through about dozen or so related answers, here are some of the findings:
Caution
Do not use below magic if you want a live graph. The graph update does not work if the notebook uses below:
%load_ext autoreload
%autoreload 2
You need below magic in your notebook before you import matplotlib:
%matplotlib notebook
Method 1: Using FuncAnimation
This has a disadvantage that graph update occurs even if your data hasn't been updated yet. Below example shows another thread updating data while Jupyter notebook updating graph through FuncAnimation.
%matplotlib notebook
from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from threading import Thread
import time
class LiveGraph:
def __init__(self):
self.x_data, self.y_data = [], []
self.figure = plt.figure()
self.line, = plt.plot(self.x_data, self.y_data)
self.animation = FuncAnimation(self.figure, self.update, interval=1000)
self.th = Thread(target=self.thread_f, daemon=True)
self.th.start()
def update(self, frame):
self.line.set_data(self.x_data, self.y_data)
self.figure.gca().relim()
self.figure.gca().autoscale_view()
return self.line,
def show(self):
plt.show()
def thread_f(self):
x = 0
while True:
self.x_data.append(x)
x += 1
self.y_data.append(randrange(0, 100))
time.sleep(1)
g = LiveGraph()
g.show()
Method 2: Direct Update
The second method is to update the graph as data arrives from another thread. This is risky because matplotlib is not thread safe but it does seem to work as long as there is only one thread doing updates.
%matplotlib notebook
from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from threading import Thread
import time
class LiveGraph:
def __init__(self):
self.x_data, self.y_data = [], []
self.figure = plt.figure()
self.line, = plt.plot(self.x_data, self.y_data)
self.th = Thread(target=self.thread_f, daemon=True)
self.th.start()
def update_graph(self):
self.line.set_data(self.x_data, self.y_data)
self.figure.gca().relim()
self.figure.gca().autoscale_view()
def show(self):
plt.show()
def thread_f(self):
x = 0
while True:
self.x_data.append(x)
x += 1
self.y_data.append(randrange(0, 100))
self.update_graph()
time.sleep(1)
from live_graph import LiveGraph
g = LiveGraph()
g.show()
I explored this and produced the following which is largely self-documenting:
import matplotlib.pyplot as plt
%matplotlib notebook
print('This text appears above the figures')
fig1 = plt.figure(num='DORMANT')
print('This text appears betweeen the figures')
fig2 = plt.figure()
print('This text appears below the figures')
fig1.canvas.set_window_title('Canvas active title')
fig1.suptitle('Figure title', fontsize=20)
# Create plots inside the figures
ax1 = fig1.add_subplot(111)
ax1.set_xlabel('x label')
ax2 = fig2.add_subplot(111)
# Loop to update figures
end = 40
for i in range(end):
ax2.cla() # Clear only 2nd figure's axes, figure 1 is ADDITIVE
ax1.set_title('Axes title') # Reset as removed by cla()
ax1.plot(range(i,end), (i,)*(end-i))
ax2.plot(range(i,end), range(i,end), 'rx')
fig1.canvas.draw()
fig2.canvas.draw()
Another simple solution, based on IPython.display functions display and clear_output. I found it here. Here is the code (based on @graham-s's answer):
from IPython.display import display, clear_output
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
m = 100
n = 100
matrix = np.random.normal(0, 1, size=(m, n))
fig = plt.figure()
ax = fig.add_subplot(111)
for i in range(m):
ax.clear()
ax.plot(matrix[i, :])
display(fig)
clear_output(wait=True)
plt.pause(0.2)
It uses %matplotlib inline instead of notebook, and does not produce small image as mentioned by @MasterScrat. Works both in Jupyter Notebook and in Jupyter Lab. Sometimes image blinks that's not very nice, but usable for quick investigations.
If you need to keep axes ranges between different frames, add ax.set_xlim/ax.set_ylim after ax.clear().
With a moderate modification of @Shital Shah's solution, I've created a more general framework which can simply apply to various scenario:
import matplotlib
from matplotlib import pyplot as plt
class LiveLine:
def __init__(self, graph, fmt=''):
# LiveGraph object
self.graph = graph
# instant line
self.line, = self.graph.ax.plot([], [], fmt)
# holder of new lines
self.lines = []
def update(self, x_data, y_data):
# update the instant line
self.line.set_data(x_data, y_data)
self.graph.update_graph()
def addtive_plot(self, x_data, y_data, fmt=''):
# add new line in the same figure
line, = self.graph.ax.plot(x_data, y_data, fmt)
# store line in lines holder
self.lines.append(line)
# update figure
self.graph.update_graph()
# return line index
return self.lines.index(line)
def update_indexed_line(self, index, x_data, y_data):
# use index to update that line
self.lines[index].set_data(x_data, y_data)
self.graph.update_graph()
class LiveGraph:
def __init__(self, backend='nbAgg', figure_arg={}, window_title=None,
suptitle_arg={'t':None}, ax_label={'x':'', 'y':''}, ax_title=None):
# save current backend for later restore
self.origin_backend = matplotlib.get_backend()
# check if current backend meets target backend
if self.origin_backend != backend:
print("original backend:", self.origin_backend)
# matplotlib.use('nbAgg',warn=False, force=True)
plt.switch_backend(backend)
print("switch to backend:", matplotlib.get_backend())
# set figure
self.figure = plt.figure(**figure_arg)
self.figure.canvas.set_window_title(window_title)
self.figure.suptitle(**suptitle_arg)
# set axis
self.ax = self.figure.add_subplot(111)
self.ax.set_xlabel(ax_label['x'])
self.ax.set_ylabel(ax_label['y'])
self.ax.set_title(ax_title)
# holder of lines
self.lines = []
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.close()
def close(self):
# check if current beckend meets original backend, if not, restore it
if matplotlib.get_backend() != self.origin_backend:
# matplotlib.use(self.origin_backend,warn=False, force=True)
plt.switch_backend(self.origin_backend)
print("restore to backend:", matplotlib.get_backend())
def add_line(self, fmt=''):
line = LiveLine(graph=self, fmt=fmt)
self.lines.append(line)
return line
def update_graph(self):
self.figure.gca().relim()
self.figure.gca().autoscale_view()
self.figure.canvas.draw()
With above 2 class, you can simply reproduce @Graham S's example:
import numpy as np
m = 100
n = 100
matrix = np.random.normal(0,1,m*n).reshape(m,n)
with LiveGraph(backend='nbAgg') as h:
line1 = h.add_line()
for i in range(0,100):
line1.update(range(len(matrix[i,:])), matrix[i,:])
Note that, the default backend is nbAgg, you can pass other backend like qt5Agg. When it is finished, it'll restore to your original backend.
and @Tom Hale's example:
with LiveGraph(figure_arg={'num':'DORMANT2'}, window_title='Canvas active title',
suptitle_arg={'t':'Figure title','fontsize':20},
ax_label={'x':'x label', 'y':''}, ax_title='Axes title') as g:
with LiveGraph() as h:
line1 = g.add_line()
line2 = h.add_line('rx')
end = 40
for i in range(end):
line1.addtive_plot(range(i,end), (i,)*(end-i))
line2.update(range(i,end), range(i,end))
Also, you can update particular line in the additive plot of @Tom Hale's example:
import numpy as np
with LiveGraph(figure_arg={'num':'DORMANT3'}, window_title='Canvas active title',
suptitle_arg={'t':'Figure title','fontsize':20},
ax_label={'x':'x label', 'y':''}, ax_title='Axes title') as g:
line1 = g.add_line()
end = 40
for i in range(end):
line_index = line1.addtive_plot(range(i,end), (i,)*(end-i))
for i in range(100):
j = int(20*(1+np.cos(i)))
# update line of index line_index
line1.update_indexed_line(line_index, range(j,end), (line_index,)*(end-j))
Note that, the second for loop is just for updating a particular line with index line_index. you can change that index to other line's index.
In my case, I use it in machine learning training loop to progressively update learning curve.
import numpy as np
import time
# create a LiveGraph object
g = LiveGraph()
# add 2 lines
line1 = g.add_line()
line2 = g.add_line()
# create 2 list to receive training result
list1 = []
list2 = []
# training loop
for i in range(100):
# just training
time.sleep(0.1)
# get training result
list1.append(np.random.normal())
list2.append(np.random.normal())
# update learning curve
line1.update(np.arange(len(list1)), list1)
line2.update(np.arange(len(list2)), list2)
# don't forget to close
g.close()
In addition to @0aslam0 I used code from here. I've just changed animate function to get next row every next time. It draws animated evolution (M steps) of all N points.
from IPython.display import HTML
import numpy as np
from matplotlib import animation
N = 5
M = 100
points_evo_array = np.random.rand(M,N)
# First set up the figure, the axis, and the plot element we want to animate
fig = plt.figure()
ax = plt.axes(xlim=(0, M), ylim=(0, np.max(points_evo_array)))
lines = []
lines = [ax.plot([], [])[0] for _ in range(N)]
def init():
for line in lines:
line.set_data([], [])
return lines
def animate(i):
for j,line in enumerate(lines):
line.set_data(range(i), [points_evo_array[:i,j]])
return lines
# call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.FuncAnimation(fig, animate,np.arange(1, M), init_func=init, interval=10, blit=True)
HTML(anim.to_html5_video())
Hope it will be useful
Here is a library that deals with real-time plotting/logging data (joystick), although I am not sure it is working with jupyter. You can install it using the usual pip install joystick.
Hard to make a working solution without more details on your data. Here is an option:
import joystick as jk
import numpy as np
class test(jk.Joystick):
# initialize the infinite loop decorator
_infinite_loop = jk.deco_infinite_loop()
def _init(self, *args, **kwargs):
"""
Function called at initialization, see the docs
"""
# INIT DATA HERE
self.shape = (10, 4) # M, N
self.data = np.random.random(self.shape)
self.xaxis = range(self.shape[1])
############
# create a graph frame
self.mygraph = self.add_frame(
jk.Graph(name="TheName", size=(500, 500), pos=(50, 50),
fmt="go-", xnpts=self.shape[1], freq_up=5, bgcol="w",
xylim=(0, self.shape[1]-1, None, None)))
@_infinite_loop(wait_time=0.5)
def _generate_fake_data(self): # function looped every 0.5 second
"""
Loop starting with the simulation start, getting data and
pushing it to the graph every 0.5 seconds
"""
# NEW (RANDOM) DATA
new_data = np.random.random(self.shape[1])
# concatenate data
self.data = np.vstack((self.data, new_data))
# push new data to the graph
self.mygraph.set_xydata(self.xaxis, self.data[-1])
t = test()
t.start()
t.stop()
t.exit()
This code will create a graph that is auto-updating 5 times a second (freq_up=5), while new data is (randomly) generated every 0.5 seconds (wait_time=0.5) and pushed to the graph for display.
If you don't want the Y-axis to wiggle around, type t.mygraph.xylim = (0, t.shape[1]-1, 0, 1).
I don't know much about matplotlib or jupyter. However, Graphs interest me. I just did some googling and came across this post. Seems like you have to render the graph as an HTML video to see a dynamic graph.
I tried that post. This is the notebook, if you wish to try. Note that the kernel (python 2) takes sometime to build the video. You can read more about it here.
Now you want to display a graph row to row. I tried this. In that notebook, I have a dump_data with 10 rows. I randomly take one and plot them and display as video.
It was interesting to learn about jupyter. Hope this helps.
