0

I created a web (Vue.js) and desktop app (Tauri) that records my camera's video stream with getUserMedia. The camera is 4K resolution which is recognized as a webcam by CamLink. And finally for the recording part I use MediaRecorder. It would have been so beautiful if it were that simple...

Picture quality is really good but I quickly discovered that it didn't record at a consistent framerate. The videos are sometimes 29.042 fps, then 30.512 fps, etc. This makes a big problem for what I want to do later. When I open the videos in after effects it jerks.

I implemented @kaiido solution with the help of the canvas but I wanted to know if there was a solution to do it without the canvas like use only MediaRecorder since it degrades the performance of the app.

I don't want to manipulate the video. I want to have the best possible quality in a constant frame rate.

Kaiido
  • 123,334
  • 13
  • 219
  • 285
KaliaJS
  • 3
  • 1

2 Answers2

2

The frame rate should be specified on the input MediaStreamTrack, e.g in the call to getUserMedia(constraints).

const constraints = {
  video: { frameRate: { exact: 30 } }
};
const stream = await navigator.mediaDevices.getUserMedia(constraints);

Though a too restrictive constraint like that has good chances of not being supported, so you might prefer request an initial ideal value, and then try to force an exact one through MediaStreamTrack#applyConstraints().

const [track] = stream.getVideoTracks();
try {
  await track.applyConstraints(constraints);
}
catch(err) {
  // maybe try again with another value,
  // e.g the one from track.getSettings().frameRate
}
Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • Sorry for the late response. Set the framerate on track.applyConstraints work on Chrome but on safari webview (Tauri.app), it does not take into account. BUT on chrome it's weird because the framerate is constant but not standard . Although I put 25 ffmpeg -i and also on After effects it say that the file is 24.92 fps. But at least it's constant. – KaliaJS Feb 11 '23 at 12:34
1

For what it's worth, getUserMedia and MediaRecorder don't generate broadcast-quality video streams. Dropped and jittery frames come with the territory. And, trying to push 2160p resolution through them is likely to overburden the host machine if it works at all.

What you can try:

  1. Specify a data rate in the options to MediaRecorder, and specify something lower than the 2.5mbps default. This reduces the encoding machine's workload a bit.

    new MediaRecorder (stream, {videoBitsPerSecond: 1000000} );

  2. Reduce the resolution of the video. You can get better, but still not guaranteed-perfect, results with 720p resolution. This reduces the workload a lot.

  3. Reduce the frame rate. This gives a little more time to code each frame, so dropped frames are less likely.

O. Jones
  • 103,626
  • 17
  • 118
  • 172
  • Thanks for your help. Yes I know it's not ideal but I think we have no other choice but to use MediaRecorder right? The app will work on a Macbook pro m1 max. Without any other app running on it. But you have to know that real time does not interest me at all. I'd rather waste time than lose frames. I just need it to be constant (25fps because the cam is 25fps). – KaliaJS Feb 11 '23 at 12:40
  • Perfection isn't feasible. Try using a Chromium-based browser (like Chrome or the browser embedded in Electron) for best results. Whether those best results will be good enough for your application is hard to say. FWIW, there are non-browser solutions to this problem n Mac tha do work well. – O. Jones Feb 11 '23 at 12:44