Buffer as input and output for fluent-ffmpeg

Question

The below looks like a lot but a it's primarily just output.

I'm trying to take in a buffer using multer (being send the request containing the video (.mov format) through Alamofire from an iPhone) as the input before a fluent-ffmpeg conversion, then I want it to output as a buffer, and then send the result to S3. I think I'm close, but I don't think fluent-ffmpeg can have a buffer passed in. This is deployed on heroku using this buildpack: https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.... How do I pass it in correctly?

const multer = require('multer')
const upload = multer({ limits: { fieldSize: 100_000_000 } })
app.post('/create', upload.single('video'), async function (request, response, next) {
  let data = request.body
  console.log(data) // prints [Object: null prototype] { param1: '' }
  let bufferStream = new stream.PassThrough();
  console.log(request.file.buffer) // prints '<Buffer 00 00 00 14 66 74 79 70 71 74 20 20 00 00 00 00 71 74 20 20 00 00 00 08 77 69 64 65 01 0e 28 55 6d 64 61 74 21 20 03 40 68 1c 21 4e ff 3c 59 96 7c 82 ... 17718642 more bytes>'

  new ffmpeg({
    source: stream.Readable.from(request.file.buffer, { objectMode: false })
  })
  // .format('mp4')
  .on('error', function (err) {
    console.log(Error: ${err})
  })
  .on('progress', function (progress) {
    console.log("progress")
  })
  .on('end', function () {
    console.log('Formatting finished!');
    console.log("after");
  })
  .writeToStream(bufferStream);

  // Read the passthrough stream
  const buffers = [];
  bufferStream.on('data', function (buf) {
    buffers.push(buf);
  });
  bufferStream.on('end', function () {
    const outputBuffer = Buffer.concat(buffers);
  // use outputBuffer
  });
  console.log("Added.")
  response.send("Success")
});

The output is what you can see below (without .format('mp4')):

2022-09-03T13:12:24.194384+00:00 heroku[router]: at=info method=POST path="/createBusiness" host=sparrow-testing.herokuapp.com request_id=2774a4ec-e21e-4c2f-8086-460a4cba7d1d fwd="74.71.236.5" dyno=web.1 connect=0ms service=13157ms status=200 bytes=762 protocol=https
2022-09-03T13:12:24.186257+00:00 app[web.1]: [Object: null prototype] { title: '' }
2022-09-03T13:12:24.187296+00:00 app[web.1]: <Buffer 00 00 00 14 66 74 79 70 71 74 20 20 00 00 00 00 71 74 20 20 00 00 00 08 77 69 64 65 01 0e 28 55 6d 64 61 74 21 20 03 40 68 1c 21 4e ff 3c 59 96 7c 82 ... 17718642 more bytes>
2022-09-03T13:12:24.189891+00:00 app[web.1]: Added.
2022-09-03T13:12:24.891564+00:00 app[web.1]: Error: Error: ffmpeg exited with code 1: pipe:1: Invalid argument
2022-09-03T13:12:24.891570+00:00 app[web.1]: 

This output is what you see with .format('mp4'):

2022-09-03T13:17:07.380415+00:00 app[web.1]: [Object: null prototype] { title: '' }
2022-09-03T13:17:07.381335+00:00 app[web.1]: <Buffer 00 00 00 14 66 74 79 70 71 74 20 20 00 00 00 00 71 74 20 20 00 00 00 08 77 69 64 65 01 0e 28 55 6d 64 61 74 21 20 03 40 68 1c 21 4e ff 3c 59 96 7c 82 ... 17718642 more bytes>
2022-09-03T13:17:07.384047+00:00 app[web.1]: Added.
2022-09-03T13:17:07.388457+00:00 heroku[router]: at=info method=POST path="/createBusiness" host=sparrow-testing.herokuapp.com request_id=84e69ead-09b1-4668-8fc8-b9fc9d5f229d fwd="74.71.236.5" dyno=web.1 connect=0ms service=13079ms status=200 bytes=762 protocol=https
2022-09-03T13:17:08.339746+00:00 app[web.1]: Error: Error: ffmpeg exited with code 1: Conversion failed!
2022-09-03T13:17:08.339783+00:00 app[web.1]: 

My uploadFile function works correctly because I use it elsewhere--normally, I just pass in the request.file.buffer, but here it needs to be a buffer after the ffmpeg conversion

EDIT:

At Heiko's suggestion, I tried changing the multer initialization to

multer({ limits: { fieldSize: 100_000_000 }, dest: "uploads/" })

and the source I was passing in to ffmpeg to

new ffmpeg({
  source: request.file.path // request.file.path seems to be a path of a Multer-generated file, I'd assume the one I'm sending to the server
})
.format('mp4')

but it still errored out to

Error: ffmpeg exited with code 1: Conversion failed!

when the request.file was:

{
  fieldname: 'video',
  originalname: 'video',
  encoding: '7bit',
  mimetype: 'clientMime',
  destination: 'uploads/',
  filename: '08d5d3bbdcf1ac29fb97800136a306e9',
  path: 'uploads/08d5d3bbdcf1ac29fb97800136a306e9',
  size: 1567480
}

Answer 1

I give a try with your code and i get same result. After searching a while, i found an explanation.

Well, problem is around MP4 and streams.

I am not a specialist of video, but it seems that when manipulating MP4 file, ffmpeg need to seek into file ans seeking is incompatible with streams by nature.

So you need to output a filesystem file from ffmpeg (make this file temporary by removing it after using it) and then use this file according to your needs.

For having more debug information about ffmpeg error you need to modify your error event listener code like this:

// .format('mp4')
  .on('error', function (err, stdout, stderr) {
    console.log(Error: ${err})
    console.log('Stdout: %o', stdout);
    console.log('Stderr: %o', stderr);
  })
  .on('progress', function (progress) {

And you will see while executing the stderr logs of the ffmpeg process.

You can have confirmation of this here:

https://stackoverflow.com/a/39679975/8807231

https://superuser.com/a/760287

Answer 2

MP4, MOV and 3GP all contain some important metadata at the end of the file. This stems from ages predating streaming and makes these formats by default unfit for it.

The solution is to store that so called moov atom at the beginning of the file. This means you must preprocess iPhone videos in order to stream them.

With ffmpeg e.g. via FFmpegKit the option is -movflags faststart.
With AVAssetWriter/AVAssetExportSession it's the flag shouldOptimizeForNetworkUse (via).
These need to write to a file on device though, maybe you can work around that with a named pipe read by Alamofire.

VLC and iOS separately transfer the moov atom to make sense of the rest. If you could apply this strategy to ffmpeg you'd only need to find the moov atom on the phone. But there were no examples to find of ffmpeg setup this way.

Your server code seems fine, probably only needs the -movflags faststart as output option there, too. For testing upload an MKV or WEBM file which ffmpeg can read streamed out of the box.

Answer 3

Overview:

Yes you can use the tmp package where you can store the video into a specific file pass it to ffmpeg and then clean the file after all processing is completed

Note: I recommend using the tmp-promise since it make it easy to deal with promises

1. Import the package

const tmp = require("tmp-promise");

2. Set the temp dir

  // Temp directory
 const tempDir = await tmp.dir({
   unsafeCleanup: true,
 });

3. Save the file into the system using the tempDir path

   // Get The video file 
    const video = req.file;


    await fs.promises.writeFile(
      `${tempDir.path}/${video.originalname}`,
      video.buffer
    );

4. Process the video

here you can pass the file to the ffmpeg and process the video as you like with your specific need

note: file name is ${tempDir.path}/${video.originalname}

5. Clean up the directory

 tempDir.cleanup()