2

I am writing a Javascript App that intends to play live video coming as a fragmented mp4 file from a server. The intention is to play it as soon as it is received so simulating a live broadcast.

The setup code is:

myMediaSource = new MediaSource();
myVideo = document.getElementById('fragvideo');
myVideo.src = window.URL.createObjectURL(myMediaSource);
myMediaSource.addEventListener('sourceopen', function(e) {
    mySourceBuffer = myMediaSource.addSourceBuffer('video/mp4; codecs="avc1.42E01E, mp4a.40.2"');
}, false);

The code which receives over websocket binary data is this:

`   function onmessage (event) {
        if(event.data instanceof ArrayBuffer)
        {
            var bytes = new Uint8Array(event.data);
            mySourceBuffer.appendBuffer(bytes);
            setTimeout(function(){
                sendmessage();
            }, 50);
        }
    }

I am sending the file from server in 4K packets at a time. I don' get any error on Javascript side but the video does not play (the video are is black).

What am I doing wrong? Is the way I am feeding the data wrong? Hoping someone can help here...

Thanks.

user3517496
  • 21
  • 1
  • 3

2 Answers2

0

Sorry to reply after a long time. It is possible that the problem is with MP4 file. At the moment Chrome is the only browser supporting MSE, and it relies on the fact that the MP4 file must comply with very strict requirements.

See for example this answer: Unable to get MediaSource working with mp4 format in chrome

Note that you should be able to check errors generated by the code playback engine if you open the chrome:media-internals and check the media blob information. The errors field will tell you what's wrong with it.

Community
  • 1
  • 1
saste
  • 710
  • 2
  • 11
  • 19
0

You can't send the file chunked with a fixed size, you need to add 'segments' of the correct format to the sourceBuffer.

In this case, you need to parse the outgoing boxes that make your fragmented MP4 and send them in pairs of moof+mdat.

Also, the first segment that you need to add is an 'intialization segment' made of an empty (as in there are no samples) moov that initializes the video engine in the browser and defines the necessary metadata to decompress the stream.

Pablo Montilla
  • 2,941
  • 1
  • 31
  • 35
  • I have a similar issue ([see question](https://stackoverflow.com/questions/64433422/how-to-play-and-seek-fragmented-mp4-audio-using-mse-sourcebuffer)). So, for this to work I need to know the size of one box where one box is `moof + mdat`? How can we get this information from a fragmented MP4? – Stefan Falk Oct 25 '20 at 07:11
  • The size of this are not fixed, they depend on the compressed data you are sending. See this answer for a bit more of detail https://stackoverflow.com/a/48208160/83169 – Pablo Montilla Oct 26 '20 at 15:24