6

I'm looking at a way to implement video encoder using web browser. Youtube and Facebook already allow you to go live directly from the web browser. I'm wondering how do they do that?

There are a couple of solutions I've researched:

  • Using web socket: using web browser to encode the video (using mediarecorder api) and push the encoded video to the server to be broadcast.
  • Using WebRTC: web browser as a WebRTC peer and another server as the other end to receive the stream and re-broadcast (transcode) using other means (rtmp, hls).

Is there any other tech to implement this that those guys (YouTube, Facebook) are using? Or they also use one of these things?

Thanks

nhong
  • 135
  • 1
  • 2
  • 11
  • i'd think websocket is used, webrtc is for peer to peer direct but last time i checked it wasn't guaranteed so you'd need to setup intermediary server called TURN? server. which would act like relay. Take a look at discord websocket scaling – Muhammad Umer Apr 03 '19 at 01:07
  • @MuhammadUmer WebRTC can be used to connect between a browser and a server just fine. The server is just one of the "peers" in this scenario. – Brad Apr 03 '19 at 01:09
  • yea to server, but not to other users if they are behind NAT. server works because you provide url to connect to just like in websocket. – Muhammad Umer Apr 03 '19 at 01:13
  • Duplicated to https://stackoverflow.com/questions/56238148/how-to-use-webrtc-to-stream-video-to-rtmp – Winlin Mar 08 '23 at 10:46

2 Answers2

5

WebRTCHacks has a "how does youtube use webrtc" post here which examines some of the technical details of their implementation.

In addition one of their engineers gave a Talk at WebRTC Boston describing the system which is available on Youtube

Philipp Hancke
  • 15,855
  • 2
  • 23
  • 31
3

Correct, you've hit on two ways to do this. (Note that for the MediaRecorder method, you can use any other method to get the data to the server. Web Sockets is one way... so is a regular HTTP PUT of segments. Or, you could even use a data channel of a WebRTC connection to the server.)

Pretty much everyone uses the WebRTC method, as there are some nice built-in benefits:

  • Low latency (at the cost of some quality)
  • Dynamic bitrate
  • Well-optimized on the client
  • Able to automatically scale output if there are not enough system resources to continue encoding at a higher frame size

The downsides of the WebRTC method:

  • Ridiculously complicated stack to maintain server-side.
  • Lower quality (due to emphasis on low latency, BUT you can tweak this by fiddling with the SDP yourself)

If you go the WebRTC route, consider gstreamer. If you want to go the Web Socket route, I've written a proxy to receive the data and send it off to FFmpeg to be copied over to RTMP. You can find it here: https://github.com/fbsamples/Canvas-Streaming-Example

Brad
  • 159,648
  • 54
  • 349
  • 530
  • 1
    i did one project in webrtc and it was most difficult thing ever. Even if you accepted counter intuitiveness of it all and jumped in, the documentation were wrong on MDN itself, wrong as in they were right according to specification, but incorrect to what browsers implemented. – Muhammad Umer Apr 03 '19 at 01:16
  • @MuhammadUmer The bigger issue here is receiving the video data server-side in a way that it can be manipulated or reencoded for distribution.. The two ways I've done this before is by using gstreamer, or by using an actual browser engine server-side. We're still lacking a magical tool that can be a proper WebRTC peer and dump frames or even the media bitstreams. – Brad Apr 03 '19 at 01:18
  • Thanks Brad, however from the video encoding aspect, WebRTC also uses the same mechanism with the MediaRecorder API right? Says, regardless of WebRTC or Web Socket - that is, regardless of the transporting mechanism, the encoding tasks are the same between these two right? – nhong Apr 03 '19 at 01:18
  • 1
    @nhong Nope! That would be nice, but they're two unrelated very different and often disconnected things. They don't even use the same codecs all the time. (For instance, in Firefox you can use a baseline profile of H.264 but last I checked into it, you can't use H.264 in MediaRecorder at all! I filed a bug report... not sure if that ever got prioritized or not.) And remember, the WebRTC stack is optimized for latency whereas the MediaRecorder encoding is not. – Brad Apr 03 '19 at 01:20
  • @nhong What would be great is if all these parameters were exposed to the MediaRecorder interface so that we could choose our own latency targets, and also dynamically scale things as needed, but that's not how it's built and that's not how the spec is written. – Brad Apr 03 '19 at 01:21
  • So you mean for every browser that supports MediaRecorder API and WebRTC, there are two implementations of the encoder? Says, two libx264 things? – nhong Apr 03 '19 at 01:21
  • @nhong I don't know about *every* but yes that's possible. And, not everyone is using libx264 either. Chrome, for example, will often use hardware codecs if exposed by the system. – Brad Apr 03 '19 at 01:22
  • I didn't record the video in my project, but I believe this has been done. I remember reading up on media server... http://www.kurento.org/ . Also i think it's possible to stream directly to AWS storage from browser, im not sure if it's possible to stream directly to youtube from browser. But since live feature works on youtube. It might be possible – Muhammad Umer Apr 03 '19 at 03:05
  • @MuhammadUmer Kurento just uses WebRTC. Yes, you can upload media chunks to an S3 bucket, using a normal HTTP PUT as I said in my answer. I've written an app for doing this. You can stream directly to YouTube from a browser, but only from their site. I don't think they've opened up their WebRTC endpoints (yet). – Brad Apr 03 '19 at 03:19