1

I am looking into WebRTC samples to stream video from client to server https://webrtc.github.io/samples/

I have a text chat in Ratchet, working fine, now I need to implement a video stream to web socket, can someone help?

index.php CHAT PAGE:

<body>
     <div id="app" class="container">

    <h3 class="text-center">Chat simples utilizando Websocket, Vue.JS e PHP</h3>    
    <div id="messages">
        <div class="col s12">
            <ul class="list-unstyled" v-cloak>
                <li v-for="message in messages">
                    <span class="date" v-if="message.date">[{{ message.date }}]</span>
                    <span class="name" v-if="message.user">{{ message.user }}:</span>
                    <span class="text" :style="{ color: message.color }">
                        {{ message.text }}
                    </span>
                </li>
            </ul>
        </div>
    </div>

    <div class="row">

        <div class="col-12 col-sm-3">
            <input type="text" class="form-control" placeholder="Usuário" v-model="user">
        </div>

        <div class="col-12 col-sm-9">
            <input type="text" class="form-control" placeholder="Mensagem" v-model="text"
                   @keyup.enter="sendMessage">
        </div>    
    </div>
    </div>

<!-- webrtc need to stream this  -->    
<div id="container">
    <h1><a href="//webrtc.github.io/samples/" title="WebRTC samples homepage">WebRTC samples</a>
        <span>getUserMedia</span></h1>
        <video id="gum-local" autoplay playsinline></video>
    <button id="showVideo">Open camera</button>    
    <div id="errorMsg"></div>    
    <p class="warning"><strong>Warning:</strong> if you're not using headphones, pressing play will cause feedback.</p>    
    <p>Display the video stream from <code>getUserMedia()</code> in a video element.</p>    
    <p>The <code>MediaStream</code> object <code>stream</code> passed to the <code>getUserMedia()</code> callback is in
        global scope, so you can inspect it from the console.</p>    
    <a href="https://github.com/webrtc/samples/tree/gh-pages/src/content/getusermedia/gum"
       title="View source for this page on GitHub" id="viewSource">View source on GitHub</a>
</div>
<script src="https://webrtc.github.io/adapter/adapter-latest.js"></script>
<script src="https://webrtc.github.io/samples/src/content/getusermedia/gum/js/main.js"></script>
  <link rel="stylesheet" href="https://webrtc.github.io/samples/src/css/main.css">
<!-- webrtc -->    
<script type="text/javascript" src="js/lib/vue.min.js"></script>
<script type="text/javascript" src="js/chat.js"></script>
</body>

chat.js VUE JS CHAT

var app = new Vue({

    el: "#app",
        data: {
        user: 'user',
        text: null,
        messages: [],
        ws: null,
    },


    created: function() {

        this.connect();
    },


    methods: {
      connect: function(onOpen) {
      var self = this;
    self.ws = new WebSocket('ws://localhost:8080');
                self.ws.onopen = function() {
                self.addSuccessNotification('Conectado');
               if (onOpen) {
                    onOpen();
                }
            };

            self.ws.onerror = function() {
                self.addErrorNotification('Não foi possível conectar-se ao servidor');
            };

             self.ws.onmessage = function(e) {
                self.addMessage(JSON.parse(e.data));
            };

        },


        addMessage: function(data) {
            this.messages.push(data);
            this.scrollDown();
        },

           addSuccessNotification: function(text) {
            this.addMessage({color: 'green', text: text});
        },
        addErrorNotification: function(text) {
            this.addMessage({color: 'red', text: text});
        },

        sendMessage: function() {

            var self = this;

            if (!self.text || !self.user) {

                return;
            }


            if (self.ws.readyState !== self.ws.OPEN) {


                self.addErrorNotification('Problemas na conexão. Tentando reconectar...');

                self.connect(function() {
                    self.sendMessage();
                });


                return;
            }

              self.ws.send(JSON.stringify({
                user: self.user,
                text: self.text,
            }));

            self.text = null;

        },


        scrollDown: function() {
            var container = this.$el.querySelector('#messages');
            container.scrollTop = container.scrollHeight;
        },

    }

});

server.php Websocket server with Ratchet:

<?php

    use Ratchet\Http\HttpServer;
    use Ratchet\Server\IoServer;
    use Ratchet\WebSocket\WsServer;


    require 'vendor/autoload.php';
    require 'class/SimpleChat.php';


    $server = IoServer::factory(
        new HttpServer(
            new WsServer(
                new SimpleChat()
            )
        ),
        8080
    );

    $server->run();

if someone has a simple solution it can be made with php or any other method, not really side to side with my text chat code, but I would prefer use Ratchet on it instead of node js. Thanks in advance guys!

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Otávio Barreto
  • 1,536
  • 3
  • 16
  • 35

1 Answers1

3

There is no (good) way for getting raw video data in Javascript to send it over websockets. It would also negate video compression, unless you implement your own encoder. On the server side, a scripting language like PHP isn't suitable for handling such data either.

The way to do this is to use the browsers webRTC functionality to send an RTP video stream. That means you establish an RTCPeerConnection with a video stream attached to it. You will still need a signalling layer for the setup. Ratchet can fill that role.

Assuming the participants in your video chat transmit their video one-to-many, a point to point connection may not be desirable due to clients bandwidth restrictions. In that case, you will need a media server software, which specializes in distributing RTP data. There are several options out there, a web search yielded this slightly aged summary: https://webrtc.ventures/2017/11/a-guide-to-webrtc-media-servers-open-source-options/

To answer your actual question: You'd have to copy the video data onto a canvas and read it from there. (Get raw pixel data from HTML5 video) Then you have a raw bitmap you could send over websockets, but you'd be looking at height x width x color-depth bytes per single video frame. For HD video in true color that would be 6 megabytes per frame, at 30fps that's 178 megabytes per second. If you don't have 1.5 gigabit pipes at the client, you'd need to encode the video in real-time, for which a single-threaded JavaScript engine is not sufficient.

Mantriur
  • 986
  • 7
  • 20