How to add matplotlib output to a video

Question

I'm trying to create a video with annotation (currently) formed from matplotlib, with the original video on the left and a FFT of some parameters on the right.

The only way I've gotten it to work is by saving a .png file for each frame, which seems tedious. I was hoping someone could point out the 'correct' method.

import cv2
import numpy as np
import matplotlib
import scipy.fftpack
from moviepy.editor import VideoFileClip
from moviepy.editor import AudioFileClip

vid = VideoFileClip('VID.mp4')
aud = AudioFileClip('VID.mp4')
out  = cv2.VideoWriter('1234.avi',cv2.VideoWriter_fourcc('M','J','P','G'), vid.fps, (vid.w*2, vid.h))
audIndex = 0
vidIndex = 0
numberOfSamples = 600
sampleRate = 800;
T = 1.0 / sampleRate;
x = np.linspace(0.0, numberOfSamples*T, numberOfSamples)

for frame in vid.iter_frames():

    # Put the recorded movie on the left side of the video frame
    frame2 = np.zeros((frame.shape[0], 2*frame.shape[1], 3)).astype('uint8')
    frame2[:720, :1280,:] = frame


    # Put, say, a graph of the FFT on the right side of the video frame
    y = np.sin(50.0 * 2.0*np.pi*x) + 0.5*np.sin(80.0 * 2.0*np.pi*x)
    yf = scipy.fftpack.fft(y)
    xf = np.linspace(0.0, 1.0/(2.0*T), numberOfSamples/2)
    fig, ax = matplotlib.pyplot.subplots()
    ax.plot(xf, 2.0/numberOfSamples * np.abs(yf[:numberOfSamples//2]))
    matFigureForThisFrame = ????????

    # Put the FFT graph on the left side of this video frame
    frame2[720:, 1280:, :] = matFigureForThisFrame

    out.write(frame2)
    vidIndex = vidIndex+1;

out.release()
#cv2.destroyAllWindows()

score 1 · Answer 1 · answered May 08 '19 at 19:07

You could try to take the path of writing to a video file directly, but I wouldn't recommend it (see here why). Writing to video file is more complicated than just changing the frames, you need to need to get the right coders and other painful issues. Personally, I would settle. Some options:

1) Generate the pngs, and afterwards concatenating them to a video file using ffmpeg

2) Save each frame to a buffer, and generate .gif file afterwards, directly in python (so you don't have to run multiple things). See this stackoverflow question on how to do that.

How to add matplotlib output to a video

1 Answers1