6

I've built some code that will get the MediaRecorder API to capture audio and video, and then use the ondataavailable function to send the corresponding webm file blobs up to a server via websockets. The server then sends those blobs to a client via websockets which puts the video together in a buffer using the Media Source Extension API.

This works well, except that if I want to start a stream partway through, I can't just send the latest blob because the blob by itself is unplayable. Also, if I send the blobs out of order the browsers usually complain that the audio encoding doesn't match up.

I really don't know as much about video containers, codecs etc as I should to pull this off, but my question is, how can I play those blobs as standalone videos? Can I somehow use code to add the information which is in the first blob (playable on its own) onto the other blobs? What would be a good approach to being able to get the stream playing partway through? I would transcode but it seems to take too long since I want to set up real-time (or close to) streaming.

Thanks!

Anthony
  • 13,434
  • 14
  • 60
  • 80

2 Answers2

3

Only the first blob received from the MediaRecorder API contains a header. So you will need to simply extract it and prepend it to your other blobs to be able to play them as standalone WebM videos. I recommend you to verify whether it works by using tools like hex editor. And you can automate this process on your server.

dubucha
  • 1,027
  • 10
  • 16
  • @SumanBogati Sorry, I can't find working code. I would recommend you try it manually and verify each chunk is playable as standalone using hex editor. Slice the header and prepend to the next chunk (backend). But I found that this approach is wrong as it starts stacking delays. TCP WebSocket is not for streaming. Try this approach - RTCPeerConnection. They already made UDP connections for you. https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection If it is for production, use Twilio. I was able to build flawless video streaming in 10 minutes. https://www.twilio.com/video – dubucha Jun 05 '20 at 21:08
  • thanks for the reply, can you please suggest something here, https://stackoverflow.com/questions/62236838/how-to-play-webm-files-individually-which-are-created-by-mediarecorder – Suman Bogati Jun 06 '20 at 19:17
2

With MSE, you can load the first chunk containing the WebM segment with track info and what not, and then start loading a cluster later on. The browser will figure it out.

WebM clusters begin with timestamps, which enable this to work.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • I got this working on Chrome, but not FireFox. Ended up going another direction and getting Kurento set up as a media server and using WebRTC which has been much easier than fiddling with MSE – Anthony Dec 16 '17 at 03:41
  • Is MSE the only way to play chuncked webm? Can't we just do like an audio/mpeg http chunked? (which plays fine over simple http) – Keyne Viana Feb 20 '18 at 18:34
  • @KeyneViana Yes, you can play it straight from a server too. The question is about data from the MediaRecorder. – Brad Feb 20 '18 at 18:47
  • @Brad I'm trying to do exactly this https://stackoverflow.com/questions/48891897/is-mediasourceextensions-the-only-way-to-play-a-chuncked-webm Mind dropping a line there? – Keyne Viana Feb 20 '18 at 18:50