Convert TensorBoard Image Sequence to Video/GIF

Question

I have a script that generates a sequence of matplotlib figures and writes them to disk using a TensorBoard SummaryWriter(). TensorBoard offers the ability to move a little slider to advance forwards and backwards through the image sequence, but I'd like to convert the image sequence to a video or animation. Is there a way to do this?

Edit 1: Here's a simplified example of what my code current does. I'd like to take the images that are written to the log file by .add_figure and convert them to a gif.

import matplotlib.pyplot as plt
import numpy as np
from torch.utils.tensorboard import SummaryWriter

n = 200
nframes = 25
x = np.linspace(-np.pi*4, np.pi*4, n)
tensorboard_writer = SummaryWriter()

for i, t in enumerate(np.linspace(0, np.pi, nframes)):
    plt.plot(x, np.cos(x + t))
    plt.plot(x, np.sin(2*x - t))
    plt.plot(x, np.cos(x + t) + np.sin(2*x - t))
    plt.ylim(-2.5,2.5)
    fig = plt.gcf()
    tensorboard_writer.add_figure(
        tag='temp',
        figure=fig,
        global_step=i,
        close=True)

You can use imageio or matplotlib’s `FuncAnimation` although I would recommend you don’t write the figures to disk between creating them and passing them to the animator unless you want to save the individual frames — William Miller, Jan 23 '20 at 19:51
`FuncAnimation` requires I store the images in memory simultaneously, right? — Rylan Schaeffer, Jan 23 '20 at 19:55
Any conversion from a set of frames to a single video file requires all the frames to simultaneously be stored in memory at some point in the process. Writing to the disk in between creates redundant reading, unless you want the individual frames preserved as well as the animated file — William Miller, Jan 23 '20 at 20:00
Yeah, I understand that writing and reading back into memory entails redundant writing and reading, but that's what I'm looking for. — Rylan Schaeffer, Jan 23 '20 at 20:12
Okay, in which case you can still use either imageio or matplotlib’s `FuncAnimation`. I will create an example to demonstrate — William Miller, Jan 23 '20 at 20:15

score 2 · Accepted Answer · answered Jan 24 '20 at 02:50

You can use either imageio or matplotlib's FuncAnimation to create an animation from a set of files representing the frames. As an example, I have created a set of files according to

import matplotlib.pyplot as plt
import numpy as np

n = 200
nframes = 25
x = np.linspace(-np.pi*4, np.pi*4, n)

for i, t in enumerate(np.linspace(0, np.pi, nframes)):
    plt.plot(x, np.cos(x + t))
    plt.plot(x, np.sin(2*x - t))
    plt.plot(x, np.cos(x + t) + np.sin(2*x - t))
    plt.ylim(-2.5,2.5)
    plt.savefig('frame.'+str(i)+'.png', bbox_inches='tight', dpi=300)
    plt.clf()

which creates a series of nframes (25 for the example) files with the naming convention 'frame.n.png' where n is the frame number. Two propagating sinusoidal waves and the created wave are being plotted over one half cycle.

With the FuncAnimation approach you then do

import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

fig = plt.figure(figsize=(10, 6), dpi=300)
nframes = 25

def animate(i):
    im = plt.imread('frame.'+str(i)+'.png')
    plt.imshow(im)
    plt.axis('off')

anim = FuncAnimation(fig, animate, frames=nframes, interval=(2000.0/nframes))
anim.save('output.gif', writer='imagemagick')

With the imageio approach you simply do

import imageio

nframes = 25
files = ['frame.'+str(i)+'.png' for i in range(nframes)]

frames = [imageio.imread(f) for f in files]
imageio.mimsave('output.gif', frames, fps=(nframes / 2.0))

Either approach produces this animation:

However, the imageio method is much faster compared to the FuncAnimation method:

> $ time python3 imageio.py
real    0m9.483s
user    0m9.484s
sys     0m1.156s

> $ time python3 FuncAnimation.py
real    15m36.151s
user    3m28.375s
sys     12m3.578s

It is also worth noting, however, that the file generated by the FuncAnimation approach is much smaller than the one created by the imageio approach.

2.5M Jan 23 18:36 FuncAnimation.gif
 13M Jan 23 18:08 imageio.gif

You can of course also use ffmpeg to do this if that is your preference over a programmatic approach.

Wow! Thanks for all this. Critically, though, my question concerns images that are written to a TensorBoard log file. Can you amend your answer to demonstrate reading the images from such a file? — Rylan Schaeffer, Jan 24 '20 at 18:12
To be clear, by TensorBoard log file, I mean the type of file written by a `SummaryWriter()` — Rylan Schaeffer, Jan 24 '20 at 18:13
@RylanSchaeffer Can you add the code you're using to write the images to the file via `SummaryWriter`? This will make it easier for me to answer your follow up question — William Miller, Jan 26 '20 at 23:49
Done! I used your code to make it as simple as possible. After the above code runs, there'll be a `runs/` directory on disk with the tensorboard log file. — Rylan Schaeffer, Jan 27 '20 at 01:53
@RylanSchaeffer The easiest way to do that would be to make the animation and write the files to disk simultaneously - instead of making the tensorboard file and then reading the figures back in to make the animation. Given the example in your edit that would be the best approach - is there any reason that wouldn't be possible with your actual code? — William Miller, Feb 03 '20 at 00:41
It's doable. It's just a pain. I had hoped that since Tensorboard offers the ability to slide through images sequentially, there might be an easy way to convert the image sequence to a gif. — Rylan Schaeffer, Feb 03 '20 at 21:39
@RylanSchaeffer It's certainly doable, but if you have the code which generates the tensorboard file then it makes far more sense to generate the animation at the same time - I'm wondering if there is any reason that wouldn't work for your use case? — William Miller, Feb 03 '20 at 21:40
The tensorboard file is written to during model training. The gif would need to be generated at the end. — Rylan Schaeffer, Feb 03 '20 at 21:44

Convert TensorBoard Image Sequence to Video/GIF

1 Answers1