Node.js merge audio and video streams and pipe it to the client

Question

I am working with ytdl-core library and it cannot download high quality videos with audio included because youtube has them in sperate files. Therefore, I need to download audio and video seperately then merge them using ffmpeg. An example of this can be seen here. However, using this way I am required to download the files prior to merging them and I was wondering is there is a way to merge audio and video streams and send the result to the client directly?

If you believe there is a more efficent way to achieve this, I would like to hear your approach.

Thanks in advance.

ffmpeg , on input side , may allow 1 input to be a stream ( STDIN ) but im pretty sure u will have to dwnlod to a blob / file either the audio trk or the video trk. Keep it simple by just downloading both to /tmp and running ffmpeg inputs from /tmp — Robert Rowntree, Aug 08 '20 at 02:56

score 3 · Answer 1 · answered Mar 06 '22 at 20:50

This worked for me... I used this video as refrence - www.youtube.com/watch?v=LoaG5G5L8x0

res.header("Content-Disposition", `attachment;  filename=.mp4`)
let video = ytdl(url,{filter:'videoonly'})
let audio = ytdl(url, {filter: 'audioonly', highWaterMark: 1<<25});

const ffmpegProcess = cp.spawn(ffmpeg, [
    '-i', `pipe:3`,
    '-i', `pipe:4`,
    '-map','0:v',
    '-map','1:a',
    '-c:v', 'copy',
    '-c:a', 'libmp3lame',
    '-crf','27',
    '-preset','veryfast',
    '-movflags','frag_keyframe+empty_moov',
    '-f','mp4',
    '-loglevel','error',
    '-'
], {
    stdio: [
    'pipe', 'pipe', 'pipe', 'pipe', 'pipe',
    ],
});

video.pipe(ffmpegProcess.stdio[3]);
audio.pipe(ffmpegProcess.stdio[4]);
ffmpegProcess.stdio[1].pipe(res);

let ffmpegLogs = ''

ffmpegProcess.stdio[2].on(
    'data',
    (chunk)=>{ffmpegLogs += chunk.toString()}
)

ffmpegProcess.on(
    'exit',
    (exitCode)=>{
        if(exitCode === 1){
            console.error(ffmpegLogs)
        }
    }
)

It works for me as I want. Thanks Mate – Muhammad Tahir Ali Apr 26 '22 at 21:27 — Muhammad Tahir Ali, Apr 26 '22 at 21:27

score 0 · Answer 2 · answered Nov 07 '20 at 18:56

I think the example u have quoted in the question will suffice your need. Instead of saving the output from stdio pipe, it can be directly piped to the response for the user to download. I have attached a sample code snippet.

app.get('/download', async (req, res)=>{
        res.header("Content-Disposition", `attachment;  filename=${videoName}.mkv`);
        const video = ytdl(url, {filter: 'videoonly'});
        const audio = ytdl(url, { filter: 'audioonly', highWaterMark: 1<<25});
        // Start the ffmpeg child process
                        const ffmpegProcess = cp.spawn(ffmpeg, [
                            // Remove ffmpeg's console spamming
                            '-loglevel', '0', '-hide_banner',
                            '-i', 'pipe:4',
                            '-i', 'pipe:5',
                            '-reconnect', '1',
                            '-reconnect_streamed', '1',
                            '-reconnect_delay_max', '4',
                            // Rescale the video
                            '-vf', 'scale=1980:1080',
                            // Choose some fancy codes
                            '-c:v', 'libx265', '-x265-params', 'log-level=0',
                            '-c:a', 'flac',
                            // Define output container
                            '-f', 'matroska', 'pipe:6',
                        ], {
                            windowsHide: true,
                            stdio: [
                            /* Standard: stdin, stdout, stderr */
                            'inherit', 'inherit', 'inherit',
                            /* Custom: pipe:4, pipe:5, pipe:6 */
                             'pipe', 'pipe', 'pipe',
                            ],
                        });
    
                        audio.pipe(ffmpegProcess.stdio[4]);
                        video.pipe(ffmpegProcess.stdio[5]);
                        ffmpegProcess.stdio[6].pipe(res); // Combining and piping the streams for download directly to the response
   }

I get an error on ```` ffmpegProcess.stdio[6].pipe(res);```` It says that its undefined — Keaton Benning, Mar 30 '21 at 15:45
Have u imported child process and ffmpeg modules, please import modules const cp = require('child_process'); const ffmpeg = require('ffmpeg-static'); Also if possible post the complete error — Vignesh Nandakumar, Apr 03 '21 at 07:39

Node.js merge audio and video streams and pipe it to the client

2 Answers2

Linked