Python ffmpeg: overlaying videos is dropping all audio

Question

I am using ffmpeg-python (source) to create the effect where you add a blurred background to fill in the sides of a tall vertical video as shown below:

The problem is that the output has no audio attached. Since the clips are the same, I want to keep the audio from one of the clips in the final output. How can I keep the audio? (I don't want to overlay the audio from both and get an echo effect, however!)

This is the function I'm using:

import ffmpeg
def add_blurred_bg():
    HEIGHT = 720 
    WIDTH = 1280
    in_file = ffmpeg.input('input.mp4')
    probe = ffmpeg.probe('input.mp4')
    video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
    iw=int(video_stream['width'])
    ih=int(video_stream['height'])
    nw = HEIGHT*iw/ih
    (
        ffmpeg
        .overlay(
            in_file.filter('scale', WIDTH, -2).crop(0,(WIDTH*HEIGHT/nw-HEIGHT)/2,WIDTH,HEIGHT).filter('gblur', sigma=40),
            in_file.filter('scale', -2, HEIGHT),
            x=(WIDTH-nw)/2
        )
        .output('output.mp4')
        .run()
    )

Thanks @Xilpex. Originally I was blowing up the video, blurring it, then cropping it. This took a long time since I would be blurring a huge video. The procedure I'm using here is from modifying this answer https://stackoverflow.com/a/54618683/5196039 — swagrov, Feb 19 '19 at 03:20

xilpex · Accepted Answer · 2019-02-18T20:32:41.677

Do you have to do it keep the audio style? Can you mix the audio and video? If so, this is a messy, but functioning example:

import ffmpeg
import os

def add_blurred_bg():
    HEIGHT = 720 
    WIDTH = 1280
    inp = 'input.mp4'
    os.system("ffmpeg -i " + inp + " -f mp3 -ab 192000 -vn music.mp3")
    print("extracting audio...")
    in_file = ffmpeg.input(inp)
    probe = ffmpeg.probe('input.mp4')
    video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
    iw=int(video_stream['width'])
    ih=int(video_stream['height'])
    nw = HEIGHT*iw/ih
    (
       ffmpeg
       .overlay(
            in_file.filter('scale', WIDTH, -2).crop(0,(WIDTH*HEIGHT/nw-HEIGHT)/2,WIDTH,HEIGHT).filter('gblur', sigma=40),
            in_file.filter('scale', -2, HEIGHT),
            x=(WIDTH-nw)/2
        )
        .output('outputPartial.mp4')
        .run()
    )
    print("bluring...")
    os.system("ffmpeg -i outputPartial.mp4 -i music.mp3 -shortest -c:v copy -c:a aac -b:a 256k output.mp4")
    print("mixing...")
    os.remove("outputPartial.mp4")
    os.remove("music.mp3")
    print("cleaning up...")
    print("done!")

I don't know why you have that problem, but here is a workaround.

STEP 1: Extract the music

STEP 2: Blur the video

STEP 3: Mix the audio and the video

STEP 4: Clean-Up

This worked for me. Unfortunately, ffmpeg does not have all the audio features implemented yet https://github.com/kkroening/ffmpeg-python/issues/26 so I had to use your `os.system` codes. Two things to note: 1) in your first os.system call, you need to have a space before `-f`. 2) I had to change your final os.system call to `os.system("ffmpeg -i outputPartial.mp4 -i music.mp3 -shortest -c:v copy -c:a aac -b:a 256k output.mp4")` Since the line you gave me was giving this error "Error initializing output stream 0:1 -- Conversion failed!" — swagrov, Feb 18 '19 at 19:57

score 1 · Answer 2 · answered Aug 06 '22 at 20:07

The other answer will work, but there's a less complex approach that will fix the audio.

Change this line:

.output('output.mp4')

to this:

.output(in_file.audio, 'output.mp4')

Explanation

When ffmpeg is given a -map option, it drops all audio/video streams which are not explicitly part of the map statement. Internally, ffmpeg-python uses map to implement overlays, and does not explicitly map the audio. See also FFMPEG overlaying video with image removes audio.

So, the fix is to explicitly map the input audio to the output.

Full working code:

import ffmpeg
def add_blurred_bg():
    HEIGHT = 720 
    WIDTH = 1280
    in_file = ffmpeg.input('input.mp4')
    probe = ffmpeg.probe('input.mp4')
    video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
    iw=int(video_stream['width'])
    ih=int(video_stream['height'])
    nw = HEIGHT*iw/ih
    (
        ffmpeg
        .overlay(
            in_file.filter('scale', WIDTH, -2).crop(0,(WIDTH*HEIGHT/nw-HEIGHT)/2,WIDTH,HEIGHT).filter('gblur', sigma=40),
            in_file.filter('scale', -2, HEIGHT),
            x=(WIDTH-nw)/2
        )
        .output(in_file.audio, 'output.mp4')
        .run()
    )

Python ffmpeg: overlaying videos is dropping all audio

2 Answers2

Explanation