I'm working on a music art video project for a song.
I have a directory that contains approximately 400 individual videos
- varying lengths (as short as 3 seconds and as long as 31 minutes)
- varying file types (mp4 and webm)
- varying resolutions/framerates/bitrates
The output video should consist of:
- 1 second chunks of each video
- in round-robin fashion
- until all videos are played fully or the output total video length reaches a certain limit (example, 20 minutes).
- Output video should be 1280×720 at 24fps, with no preference to bitrate.
- Videos that are larger should be scaled down and letterboxed (vertically or horizontally).
- Audio is not important at all. The video can be silent. I can overlay audio separately.
I do not want to loop short videos. In the example below, you can see that, for top-left-view-take-1.mp4, each 'clip' that's taken is incrementally further into the video. In other words, it shouldn't take the same 1 second clip from the beginning. The goal is to get further in to each individual video as the output video progresses.
For example, say my directory contains files such as the following (and, for the example here, we'll say that this is all files in the directory):
overhead-view-take-1.mp4 (3 seconds, for the sake of the question)
top-right-view-take-1.mp4 (3 seconds, for the sake of the question)
top-right-view-take-2.mp4 (5 seconds, for the sake of the question)
outside-kaleidoscope-1.mp4
yellow-kaleidoscope-1.mp4
red-kaleidoscope-1.mp4
brake-lights-slow-1.mp4
soft-city-lights.webm
Assume anything that isn't marked with a video duration is at least 5 seconds long.
Based on the above files & times, here would be the order and time code of each clip during the first 30 seconds of the output video:
Based on each video's total duration, the output video should include 1 second clips from all videos in the directory.
| Output time | Clip time | Clip file |
|---|---|---|
| 00:00-00:01 | 00:00-00:01 | overhead-view-take-1.mp4 |
| 00:01-00:02 | 00:00-00:01 | top-right-view-take-1.mp4 |
| 00:02-00:03 | 00:00-00:01 | top-right-view-take-2.mp4 |
| 00:03-00:04 | 00:00-00:01 | outside-kaleidoscope-1.mp4 |
| 00:04-00:05 | 00:00-00:01 | yellow-kaleidoscope-1.mp4 |
| 00:05-00:06 | 00:00-00:01 | red-kaleidoscope-1.mp4 |
| 00:06-00:07 | 00:00-00:01 | brake-lights-slow-1.mp4 |
| 00:07-00:08 | 00:00-00:01 | soft-city-lights.webm |
| 00:08-00:09 | 00:01-00:02 | overhead-view-take-1.mp4 |
| 00:09-00:10 | 00:01-00:02 | top-right-view-take-1.mp4 |
| 00:10-00:11 | 00:01-00:02 | top-right-view-take-2.mp4 |
| 00:11-00:12 | 00:01-00:02 | outside-kaleidoscope-1.mp4 |
| 00:12-00:13 | 00:01-00:02 | yellow-kaleidoscope-1.mp4 |
| 00:13-00:14 | 00:01-00:02 | red-kaleidoscope-1.mp4 |
| 00:14-00:15 | 00:01-00:02 | brake-lights-slow-1.mp4 |
| 00:15-00:16 | 00:01-00:02 | soft-city-lights.webm |
| 00:16-00:17 | 00:02-00:03 | overhead-view-take-1.mp4 |
| 00:17-00:18 | 00:02-00:03 | top-right-view-take-1.mp4 |
| 00:18-00:19 | 00:02-00:03 | top-right-view-take-2.mp4 |
| 00:19-00:20 | 00:02-00:03 | outside-kaleidoscope-1.mp4 |
| 00:20-00:21 | 00:02-00:03 | yellow-kaleidoscope-1.mp4 |
| 00:21-00:22 | 00:02-00:03 | red-kaleidoscope-1.mp4 |
| 00:22-00:23 | 00:02-00:03 | brake-lights-slow-1.mp4 |
| 00:23-00:24 | 00:02-00:03 | soft-city-lights.webm |
Now, the output video should include 1 second clips only from videos that still have time remaining. The videos that don't have any time remaining, such as overhead-view-take-1.mp4 and top-right-view-take-1.mp4, are not present here.
| Output time | Clip time | Clip file |
|---|---|---|
| 00:24-00:25 | 00:03-00:04 | top-right-view-take-2.mp4 |
| 00:25-00:26 | 00:03-00:04 | outside-kaleidoscope-1.mp4 |
| 00:26-00:27 | 00:03-00:04 | yellow-kaleidoscope-1.mp4 |
| 00:27-00:28 | 00:03-00:04 | red-kaleidoscope-1.mp4 |
| 00:28-00:29 | 00:03-00:04 | brake-lights-slow-1.mp4 |
| 00:29-00:30 | 00:03-00:04 | soft-city-lights.webm |
What I've tried
- I've read through the docs and cobbled together some code that produces output, but it only seems to work with images; I can't get the same thing to work with videos (specifically the incremental chunks).
- I've tried manipulating commands meant to `create a snapshot every x seconds/frames' but I've hit dead ends there.
- I've also started trying to create a text file to run the input from. That's the point where I thought it would make sense to ask here.
My main issue is picking off incremental chunks of individual videos, and playing those in sequence.
Thoughts?
Environment details I have access to Win, Mac, and Linux machines. So, that environment isn't as important to me. Here's ffmpeg's output:
ffmpeg version 4.3.1-0york0~16.04 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.12) 20160609
configuration: --prefix=/usr --extra-version='0york0~16.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libzimg --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Hyper fast Audio and Video encoder
