2

I have seen multiple questions similar to this but have not found a satisfactory answer.

Is there any possible way to extract video data from a NodeJS process? More specifically I am trying to extract frame data from videos. I am looking for a single-dimensional array of RGB data for every frame. I was using opencv4nodejs but that is extreme overkill and I am having trouble including the library with Electron.

Any ideas?

Jimmy Leahy
  • 566
  • 3
  • 14
  • See: https://stackoverflow.com/questions/667045/getpixel-from-html-canvas. Instead of looping to invert, just loop to write each pixel value into some array. – VC.One Oct 17 '19 at 06:28

1 Answers1

7

You might think that openCV is overkill but it really isn't. Getting RGB data from a video means decoding that video, and decoding a video is one of the more complex things that you can do with a computer (in the general case). If you want to be able to handle a standard H.264-encoded video inside of an mp4 container you're going to need something like GStreamer or FFMPEG and their associated libraries, OpenCV relies on LibAVCodec and LibAVFormat I believe (these are the libraries that underly FFMPEG).

There is no "simple" way to get RGB data of an arbitrary video without having a video decoding library and those are not going to be JS-native, they're going to be monolithic and written in C.

I was going to say you could look for something like this (https://github.com/fluent-ffmpeg/node-fluent-ffmpeg), but you're actually probably best off using OpenCV if you want a literal multi-dimensional array of RGB data. If you're trying to do encoding/decoding/remuxing, then take a look at the FFMPEG plugins for NodeJS.

John Allard
  • 3,564
  • 5
  • 23
  • 42