ffmpeg buffering stdin when spawned from node

Question

I'm trying to use node.js to write png data to ffmpeg to create video. I've verified that the following code works:

let ffmpeg = spawn(
  "ffmpeg",
  [
    "-y",
    "-loglevel", "info",
    "-f", "image2pipe",
    "-c:v", "png",
    "-r", `${this.fps}`,
    "-i", "-",

    "-an",
    "-vcodec", "libx264",
    "-pix_fmt", "yuv420p",
    "output.mp4",
  ],
);
ffmpeg.stdout.on('data', data => {
  console.log(`ffmpeg stdout: ${data}`);
});
ffmpeg.stderr.on('data', data => {
  console.log(`ffmpeg stderr: ${data}`);
});

let bufs = [];
let p = Promise.resolve();
for (let i = 0; i < this.frameData.length; i++) {
  p = p.then(_ => new Promise(resolve => {
    this.renderFrame(i);
    this.renderer.domElement.toBlob(blob => {
      blob.arrayBuffer().then(arr => {
        console.log('writing...');
        let uInt8Arr = new Uint8Array(arr);
        // ffmpeg.stdin.write(uInt8Arr);
        bufs.push(uInt8Arr);
        resolve();
      });
    });
  }));
}
p = p.then(_ => {
  for (let buf of bufs) {
    ffmpeg.stdin.write(buf);
  }
  ffmpeg.stdin.end();
});

Sorry if it seems complicated, the promises are used as a workaround for the fact that toBlob() runs asynchronously but the frames must be written in order.

That code works correctly as written, but it's needlessly inefficient because it writes the frame data to an array only to write it to ffmpeg later. However if I uncomment ffmpeg.stdin.write(uInt8Arr); and comment the loop where the array data is copied to ffmpeg, ffmpeg will simply hang at the end without generating any video. If I then perform a "large enough" action like save a file or spawn a new process, ffmpeg will generate the video correctly.

I suspect that there's some sort of buffering issue causing this, but I don't fully understand it or know how to fix it. I've already tried sending various signals with to ffmpeg with ffmpeg.kill() and running it through stdbuf as suggested here to no avail. Is there any workaround for this?

Guessing here, so not comfortable with posting it as an answer: have you tried adding a callback to `ffmpeg.stdin.write`? eg `ffmpeg.stdin.write(() => resolve())` (or even just `.write(resolve)` since you don't use the return value). Are you sure `p` resolves so that the `.then(_ => ffmpeg.stdin.end())` executes? — RickN, May 01 '20 at 11:13
Hello ! Out of curiosity, how do you render each frame `this.renderFrame(i)` and what does `this.renderer` represents? — Abouhassane Abdelhamid, May 17 '22 at 20:29

score 0 · Accepted Answer · answered Jun 02 '20 at 22:54

0

For anyone who chanced upon this, the solution was to replace the call to ffmpeg.stdin.close() with ffmpeg.stdin.destroy().

answered Jun 02 '20 at 22:54

devneal17

281
1
4
14

ffmpeg buffering stdin when spawned from node

1 Answers1