8

I'm currently trying to edit videos with the Python library of FFMPEG. I'm working with multiple file formats, precisely .mp4, .png and text inputs (.txt). The goal is to embed the different video files within a "layout" - for demonstration purposes I tried to design an example picture:

Example

The output is supposed to be a 1920x1080 .mp4 file with the following Elements:

  • Element 3 is the video itself (due to it being a mobile phone screen recording, it's about the size displayed there)
  • Element 1 and 2 are the "boarders", i.e. static pictures (?)
  • Element 4 represents a regularly changing text - input through the python script (probably be read from a .txt file)
  • Element 5 portrays a .png, .svg or alike; in general a "picture" in the broad sense.

What I'm trying to achieve is to create a sort of template file in which I "just" need to input the different .mp4 and .png files, as well as the text and in the end I'll receive a .mp4 file whereas my Python script functions as the navigator sending the data packages to FFMPEG to process the video itself.

I dug into the FFMPEG library as well as the python-specific repository and wasn't able to find such an option. There were lots of articles explaining the usage of "channel layouts" (though these don't seem to fit my need).

In case anyone wants to try on the same versions:

  • python --version: Python 3.7.3
  • pip show ffmpeg: Version: 1.4 (it's the most recent; on an off-topic note: It's not obligatory to use FFMPEG, I'd prefer using this library though if it doesn't offer the functionality I'm looking for, I'd highly appreciate if someone suggested something else)
J. M. Arnold
  • 6,261
  • 3
  • 20
  • 38
  • 1
    Would you accept `ffmpeg` command line example? Or do you need ffmpeg-python example specifically? – llogan Jan 05 '21 at 17:52
  • @llogan Well as my whole architecture is built upon Python, I'd appreciate a Python-based ffmpeg solution, though I'm thankful for everything I can find! To my knowledge both of the libraries don't differ as much so hopefully it's supported with Python (i.e. I would try to reproduce your approach in Python). – J. M. Arnold Jan 05 '21 at 18:14
  • @llogan So I re-checked the documentation and with the help of a pure `ffmpeg` solution I should be able to transfer it into Python.. Nearly all the functionality is provided by the Python-based ffmpeg (all well-documented), hence I'm fairly confident that this is doable. – J. M. Arnold Jan 05 '21 at 21:53
  • @llogan Might wanna give it a shot? The question is bountied now and as you're one of the only three person on whole StackOverflow to have ever achieved the gold [ffmpeg](https://stackoverflow.com/help/badges/5086/ffmpeg) badge, I'm sure you can help out! – J. M. Arnold Jan 09 '21 at 16:48
  • I have some experience with both ffmpeg and python, so i may make you a sample code before the bounty expires ! – TheEagle Jan 12 '21 at 22:52
  • @frederic Sure, I’d be happy to see your approach! – J. M. Arnold Jan 12 '21 at 22:53
  • what is the size of the main video in the center, and do the "boarders" need to be pictures, or would it be enough for you to specify a color ? – TheEagle Jan 12 '21 at 22:55
  • @frederic The side bars are pictures and the size of the main video varies, though the side pictures are the same size. In total the format shall be 1920x1080. Most of the time the middle video is a normal screen recording format for a phone. – J. M. Arnold Jan 12 '21 at 22:58
  • okay, but it wont be done before tomorrow ! – TheEagle Jan 12 '21 at 23:11
  • @frederic It's about the quality not about the speed (considering you leave a reply within the bounty period) – J. M. Arnold Jan 13 '21 at 17:48
  • its done jmarnold – TheEagle Jan 13 '21 at 17:54

2 Answers2

6

enter image description here

ffmpeg -i left.jpg -i video.mp4 -i right.png -i logo.png -filter_complex "[0]scale=(1920-1080*($width/$height))/2:1080:force_original_aspect_ratio=increase,crop=(1920-1080*($width/$height))/2:1080[left];[1]scale=-2:1080[main];[2]scale=(1920-1080*($width/$height))/2:1080:force_original_aspect_ratio=increase,crop=(1920-1080*($width/$height))/2:1080[right];[left][main][right]hstack=inputs=3[bg];[bg][3]overlay=5:main_h-overlay_h-5:format=auto,drawtext=textfile=text.txt:reload=1:x=w-tw-10:y=h-th-10,format=yuv420p" -movflags +faststart output.mp4

What this does is scale video.mp4 so it is 1080 pixels high and autoscale the width. left.jpg and right.png are scaled to take up the remainder so the result is 1920x1080.

This is a simple example and won't work for all inputs such as if the video.mp4 autoscaled width is bigger than 1920, but you can deal with that by using arithmetic expressions.

$width and $height refer to the size of video.mp4. See Getting video dimension / resolution / width x height from ffmpeg for a Python friendly method using JSON or XML.

See documentation for scale, crop, hstack, drawtext, overlay, and format filters.

A much simpler method is to just add colored bars instead of arbitrary images. See Resizing videos with ffmpeg to fit a specific size.

Change text on the fly by atomically updating text.txt. Or use the subtitles filter instead of drawtext if you want the text to change on certain timestamps (you did not specify what you needed).

Related answers:

llogan
  • 121,796
  • 28
  • 232
  • 243
  • Thanks for your reply! May you elaborate the implementation in Python a bit? As far as I understood this is just for the FFMPEG CLI, hence Python doesn't function as a sort of navigator or broadly speaking, your solution doesn't include e.g. changing text elements, pictures etc. – J. M. Arnold Jan 12 '21 at 14:49
  • @J.M.Arnold I'm not a Python user, that's why I asked if a ffmpeg cli solution would be acceptable before providing an answer. – llogan Jan 12 '21 at 17:05
  • Regarding your question above, the side bars would be both consisting of pictures. – J. M. Arnold Jan 12 '21 at 17:31
  • @J.M.Arnold 2 different pictures? Or 1 picture (same on each side)? Or 1 picture as background? – llogan Jan 12 '21 at 17:32
  • Depends, would prefer a solution for two different pictures. – J. M. Arnold Jan 12 '21 at 18:31
  • @J.M.Arnold Are the input images and video arbitrary/random sizes, or are they always a specific size? – llogan Jan 12 '21 at 18:38
  • All the videos and pictures change and hence I can't guarantee the same size; on an off-topic note could you mention how to call the ffmpeg command e.g. externally? e.g. by using Python (without the Python library)? I imagine I could use your custom made command as a boiler template and execute the batch file from Python - that should work, right? – J. M. Arnold Jan 12 '21 at 18:39
  • @J.M.Arnold Yes, that should work. You may find [examples like this one](https://stackoverflow.com/a/9428273/) if you search `[ffmpeg] [python] is:answer`. – llogan Jan 12 '21 at 20:58
  • Thanks for your suggestion as well! You shed some light on this topic for further references and well deserve the upvotes (including mine). Unfortunately, as another user has included a Python approach, I couldn't give you the check mark and/or bounty due to fairness reasons. However, I'm thankful for your insightful input and valuable knowledge you shared in this question - thanks! – J. M. Arnold Jan 14 '21 at 18:57
3

I've done it. The code can be used as a command line program or as a module. To find out more about the command line usage, call it with the --help option. For module usage, import the make_video fucntion in your code (or copy-paste it), and pass the appropriate arguments to it. I have included a screenshot of what my script produced with some sample material, and of course, the code. Code:

#!/usr/bin/python3
#-*-coding: utf-8-*-

import sys, argparse, ffmpeg, os

def make_video(video, left_boarder, right_boarder, picture_file, picture_pos, text_file, text_pos, output):
    videoprobe = ffmpeg.probe(video)
    videowidth, videoheight = videoprobe["streams"][0]["width"], videoprobe["streams"][0]["height"] # get width of main video
    scale = (1080 / videoheight)
    videowidth *= scale
    videoheight *= scale
    videostr = ffmpeg.input(video) # open main video
    audiostr = videostr.audio
    videostr = ffmpeg.filter(videostr, "scale", "%dx%d" %(videowidth, videoheight))
    videostr = ffmpeg.filter(videostr, "pad", 1920, 1080, "(ow-iw)/2", "(oh-ih)/2")
    videostr = videostr.split()
    boarderwidth = (1920 - videowidth) / 2 # calculate width of boarders
    left_boarderstr = ffmpeg.input(left_boarder) # open left boarder and scale it
    left_boarderstr = ffmpeg.filter(left_boarderstr, "scale", "%dx%d" % (boarderwidth, 1080))
    right_boarderstr = ffmpeg.input(right_boarder) # open right boarder
    right_boarderstr = ffmpeg.filter(right_boarderstr, "scale", "%dx%d" % (boarderwidth, 1080))
    picturewidth = boarderwidth - 100 # calculate width of picture
    pictureheight = (1080 / 3) - 100 # calculate height of picture
    picturestr = ffmpeg.input(picture_file) # open picture and scale it (there will be a padding of 100 px around it)
    picturestr = ffmpeg.filter(picturestr, "scale", "%dx%d" % (picturewidth, pictureheight))
    videostr = ffmpeg.overlay(videostr[0], left_boarderstr, x=0, y=0) # add left boarder
    videostr = ffmpeg.overlay(videostr, right_boarderstr, x=boarderwidth + videowidth, y=0) #add right boarder
    picture_y = (((1080 / 3) * 2) + 50) # calculate picture y position for bottom alignment
    if picture_pos == "top":
        picture_y = 50
    elif picture_pos == "center":
        picture_y = (1080 / 3) + 50
    videostr = ffmpeg.overlay(videostr, picturestr, x=50, y=picture_y)
    text_x = (1920 - boarderwidth) + 50
    text_y = ((1080 / 3) * 2) + 50
    if text_pos == "center":
        text_y = (1080 / 3) + 50
    elif text_pos == "top":
        text_y = 50
    videostr = ffmpeg.drawtext(videostr, textfile=text_file, reload=1, x=text_x, y=text_y, fontsize=50)
    videostr = ffmpeg.output(videostr, audiostr, output)
    ffmpeg.run(videostr)

def main():
    #create ArgumentParser and add options to it
    argp = argparse.ArgumentParser(prog="ffmpeg-template")
    argp.add_argument("--videos", help="paths to main videos (default: video.mp4)", nargs="*", default="video.mp4")
    argp.add_argument("--left-boarders", help="paths to images for left boarders (default: left_boarder.png)", nargs="*", default="left_boarder.png")
    argp.add_argument("--right-boarders", help="paths to images for right boarders (default: right_boarder.png)", nargs="*", default="right_boarder.png")
    argp.add_argument("--picture-files", nargs="*", help="paths to pictures (default: picture.png)",  default="picture.png")
    argp.add_argument("--picture-pos", help="where to put the pictures (default: bottom)", choices=["top", "center", "bottom"], default="bottom")
    argp.add_argument("--text-files", nargs="*", help="paths to files with text (default: text.txt)", default="text.txt")
    argp.add_argument("--text-pos", help="where to put the texts (default: bottom)", choices=["top", "center", "bottom"], default="bottom")
    argp.add_argument("--outputs", nargs="*", help="paths to outputfiles (default: out.mp4)", default="out.mp4")
    args = argp.parse_args()
    # if only one file was provided, put it into a list (else, later, every letter of the filename will be treated as a filename)
    if type(args.videos) == str:
        args.videos = [args.videos]
    if type(args.left_boarders) == str:
        args.left_boarders = [args.left_boarders]
    if type(args.right_boarders) == str:
        args.right_boarders = [args.right_boarders]
    if type(args.picture_files) == str:
        args.picture_files = [args.picture_files]
    if type(args.text_files) == str:
        args.text_files = [args.text_files]
    if type(args.outputs) == str:
        args.outputs = [args.outputs]

    for i in (range(0, min(len(args.videos), len(args.left_boarders), len(args.right_boarders), len(args.picture_files), len(args.text_files), len(args.outputs))) or [0]):
        print("Info : merging video %s, boarders %s %s, picture %s and textfile %s into %s" % (args.videos[i], args.left_boarders[i], args.right_boarders[i], args.picture_files[i], args.text_files[i], args.outputs[i]))
        # check if all files provided with the options exist
        if not os.path.isfile(args.videos[i]):
            print("Error : video %s was not found" % args.videos[i])
            continue
        if not os.path.isfile(args.left_boarders[i]):
            print("Error : left boarder %s was not found" % args.left_boarders[i])
            continue
        if not os.path.isfile(args.right_boarders[i]):
            print("Error : rightt boarder %s was not found" % args.right_boarders[i])
            continue
        if not os.path.isfile(args.picture_files[i]):
            print("Error : picture %s was not found" % args.picture_files[i])
            continue
        if not os.path.isfile(args.text_files[i]):
            print("Error : textfile %s was not found" % args.text_files[i])
            continue
        try:
            make_video(args.videos[i], args.left_boarders[i], args.right_boarders[i], args.picture_files[i], args.picture_pos, args.text_files[i], args.text_pos, args.outputs[i])
        except Exception as e:
            print(e)

if __name__ == "__main__":
    main()

Example for direct usage as a script:

$ ./ffmpeg-template --videos input1.mp4 inout2.mp4 --left-boarders left_boarder1.png left_boarder2.png --right-boarders right_boarder1.png right_boarder2.png --picture-files picture1.png picture2.png --text-files text1.txt text2.png --outputs out1.mp4 out2.mp4 --picture-pos bottom --text-pos bottom

As values for the options i took the defaults. If you omit the options, these defaults will be used, and if one of the files is not found, an error message will be displayed.

Image: enter image description here

TheEagle
  • 5,808
  • 3
  • 11
  • 39
  • @J.M.Arnold what do you mean with invalid - i can see it ... – TheEagle Jan 13 '21 at 18:28
  • Mhm okay weird bug on my side probably. Furthermore, your code - if I read correctly - only edits one video, right? It takes a video as an input, not a list of videos or as the question implies “multiple videos”, doesn’t it? – J. M. Arnold Jan 13 '21 at 18:29
  • @J.M.Arnold i embedded the image into my post and edited it to make the usage more clear ;) – TheEagle Jan 13 '21 at 18:31
  • 1
    @J.M.Arnold your question title indeed says something about "multiple videos", but your question says that you have a video (element 3), two static pictures for the boarders (elements 1 and 2), a text file to be displayed that changes regularly (element 4) and a picture to be displayed (element 5), and want to overlay these as in the image you added. Thats just what my script does. – TheEagle Jan 13 '21 at 18:44
  • read the first few sentences of my question. The goal is to process multiple videos with snipping them together and embedding them into two pictures, including a picture, text element, etc. – J. M. Arnold Jan 13 '21 at 18:46
  • @J.M.Arnold Do you mean you have say 5 videos, but only 1 text file, 1 picture, and 2 boarder pictures that are common for all videos ? – TheEagle Jan 13 '21 at 18:48
  • No, I have a list of videos - each video should be edited accordingly with text boarders, etc. -> those edited videos are supposed to be added. Each video has a corresponding text, etc. though I think that’s doable with the code you provided, just a little bit of rewriting.. – J. M. Arnold Jan 13 '21 at 18:49
  • 1
    @J.M.Arnold i updated my answer, the code now takes as input lists of videos, boarder images, pictures, text files, and output files, and merges them as described – TheEagle Jan 13 '21 at 19:29
  • Hi, Frederic sir sorry for messaging here. This a query which I had on formed with python.https://stackoverflow.com/questions/65679936/avoiding-running-of-ffmpeg-on-terminal-cmd and here u imported FFmpeg is it in python FFmpeg I mean FFmpeg-python? and does this have all the features and same syntaxes Thank you – Education 4Fun Jan 14 '21 at 12:18
  • Congratulations! Your answer does fulfil the criteria of my original asked question (including Python). The code itself could take some improvements (formatting-wise), but overall you should a considerable amount of effort and fulfilled the requirements. – J. M. Arnold Jan 14 '21 at 18:56
  • @J.M.Arnold I too did not interpret "multiple videos" to be a list or involve concatenation, but just the collection of elements as shown in the question. I assumed "multiple videos" meant that the command should work with any arbitrary video, but one at a time. I have to say there was a lot of guesswork involved in creating an answer for this question. – llogan Jan 14 '21 at 20:14
  • @J.M.Arnold what do you mean with "could take some improvements (formatting-wise)" ? – TheEagle Jan 14 '21 at 21:19
  • @llogan I appreciate your feedback - thanks a lot, I'll try to improve my question asking skills. – J. M. Arnold Jan 14 '21 at 22:45