Feeding raw image bytes into ffmpeg rawvideo fails with Invalid buffer size on linux only

Question

I have a nodejs program which generates raw (rgb24) image(s), which I then pipe into ffmpeg so it saves as png or mp4. My code looks like this:

const fs = require("fs");
// ...
const outputBuffer = Buffer.alloc(outputPngWidth * 3 * outputPngHeight);
// ... write data into outputBuffer
fs.writeSync(process.stdout.fd, outputBuffer);

I then do the following in CLI:

node generate | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 1000x1000 -i - test.png

Alternatively, if I generate lots of images from my program, I do this to generate the video file:

node generate | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 1000x1000 -r 60 -i - -codec:v libx265 test.mp4

On windows this works flawlessly. On linux (either on Ubuntu 20 VM, or Ubuntu 20 installed directly on a physical machine), it consistently fails with:

pipe:: corrupt input packet in stream 0
[rawvideo @ 0x55f5256c8040] Invalid buffer size, packet size 65536 < expected frame_size 3000000
Error while decoding stream #0:0: Invalid argument

If I split this in 2 phases like so, then it works perfectly on linux as well:

node generate > test.raw
cat test.raw | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 1000x1000 -i - test.png

By looking at the error "packet size 65536 < expected frame_size 3000000" it seems that node's fs.writeSync only sends 65536 bytes at a time, but ffmpeg expects 3000000 bytes (that is 1000 width * 1000 height * 3 channels).

If I reduce my image size to something small, e.g 50x50 or 100x100, then it works. As soon as x * y * 3 exceeds 65536, it fails (eg. 160x160 fails with "packet size 65536 < expected frame_size 76800" because 160 * 160 * 3 = 76800).

What I've tried so far to solve the issue without luck:

• Force node to spit out the whole buffer at once:

fs.writeSync(process.stdout.fd, outputBuffer, 0, outputBuffer.length);

• All suggestions of Add a big buffer to a pipe between two commands with various linux commands to buffer between node and ffmpeg.
• Use Ubuntu 18 instead of 20.
• Use node 12 instead of 15.
• Figure out a way to change the chunk size in https://nodejs.org/api/fs.html

Is there a way to overcome this?

Answer 1

This is a stab in the dark and I know the following might sound outlandish.

Seeing the error message from the tests, it seems that ffmpeg times out while it is waiting for sufficient data and the generator does not fill the pipe fast enough at the same time.

I asked myself how ffmpeg could obviously timeout listening to STDIN although it - hopefully - would use a blocking read call? Well, perhaps it does not make a blocking call to read but calls select on the file descriptor with a short timeout instead and then just runs into that timeout while waiting for sufficient data.

A neat explanation how this could happen is if the system is kept busy either with generating data or with consuming the data, i.e. the counterpart is always idle.

Since you say that it is a VM that is running all of this, I could imagine that the VM is configured to have only a single CPU available. It can then execute only one of the two processes at any given time, the processes compete for the CPU.

I can see this being supported by the other test you ran: you created the test data first and stored it in a file. Then you let ffmpeg consume it. Each process can use the available CPU full time. no competition.

Also a multi-CPU or multi-Hyperthreading system could expose this behaviour if it is just kept busy enough with other processes that have higher priority.

Bottom line: I would make sure that my VM has enough processing power for both processes.