0

Wasn't sure how to word this exactly, but basically I want to make a web app that allows people to make basic key frame animations of elements in a canvas, similar in concept to after effects or flash etc, and then click "render", which would then start playing the keyframes etc. into the media recorder api in order to render it to an HD video, but not necessarily realtime, rather, one frame at a time, by it has to be an exact frame rate, so the audio matches up with the video etc.

Basically, like after effects or blender rendering in the browser, how do I use the MediaRecorder API. (or perhaps ffmpeg.js) to record an HD video based on pre designed canvas animations, at an exact framerate (even if some frames take longer to render than others, but the video, finished product should be a set framerate)?

I'm aware of the canvas.captureStream(frame rate here), then I simply pass that stream to the media recorder api, and start playing the animation of the canvas etc,

For example if the desired final video frame rate is 30 FPS, then I could capture the canvas at 15 FPS and play the keyframes back with JavaScript also at 15 FPS, although I think I'm missing something, because I don't think that would result in the final video being 30FPS, and some frames might take a little longer etc, but it has to be an exact frame rate

Is there a way to do the kind of rendering that after effects and blender does with client side JavaScript, either using the media recorder API or something else? How can I guarantee a set frame rate?

John Goshen
  • 139
  • 5
  • If you want to record a 30fps video, you'd capture at 30fps regardless of the frame rate fed into the canvas. So, `canvas.captureStream(30)`. – Ouroborus Apr 21 '21 at 23:22
  • @ouroborus What do I do about slow devices then that can't that fast? – John Goshen Apr 22 '21 at 01:35
  • If the device is slow enough that it can't handle it that way, it's going to be even slower in assembling a video frame by frame. Generally browsers aren't meant for that kind of workload on their own. – Ouroborus Apr 22 '21 at 02:29
  • @ouroborus great, even slower is fine with me, as long as it gets a functioning video eventually. How can I do this? Are you telling me it's absolutely completely impossible to do this in a browser? – John Goshen Apr 22 '21 at 02:35
  • It's possible. For example there's a nodejs package [mp4-mjpeg](https://www.npmjs.com/package/mp4-mjpeg) that looks like it does what you want. Since it's designed for node, you'd have to rewrite parts of it so that it could work in the browser. (Some of node's libraries, like `fs`, aren't available in the browser but also aren't strictly necessary for the encoding.) It looks like it only accepts jpgs so you'd need something else to convert the canvas data to a jpeg. – Ouroborus Apr 22 '21 at 04:38
  • @ouroborus wow that's actually fascinating, only about 15kb of source code (besides 2 dependencies) to make mp4s in JavaScript, it's fairly trivial to convert a canvas to a jpg, the only problem would be doing so at a consistent frame rate, but it should theoretically be possible – John Goshen Apr 23 '21 at 16:12

0 Answers0