20

I'm looking for a solution to play raw h264 stream coming from a native server through WebSocket live on a browser. I tried many third party h264 decoders in JavaScript and each one has its own issues. Decoders based on broadway cannot decode main and high profile h264. Other decoders are too slow to decode 1080p frames. I tried converting raw h264 into fragmented mp4 in JavaScript but the playback is very ugly when decoding bidirectional frames. I also tried webrtc but it seems impossible to implement peer-connection between browser and a native server. Any suggestions?

Dale K
  • 25,246
  • 15
  • 42
  • 71
Kiran Raj
  • 201
  • 1
  • 2
  • 4
  • 1
    You will need to mux your h.264 stream into an MP4/ISO BMFF container, and then use MediaSource Extensions to play it back. You mentioned you tried this... can you show the code you tried? – Brad Jan 02 '19 at 17:25
  • Yes i tried it. I used this project https://github.com/xevokk/h264-converter to do the muxing. It does not play well in chrome due to the fact that chrome decoder refers decoding timestamp instead of presentation timestamp for decoding b-frames. – Kiran Raj Jan 03 '19 at 13:46
  • MSE in Chrome can handle b-frames just fine (unlike WebRTC), you just need to provide composition times in your fragmented mp4. So whatever library you are using for muxing, make sure to set composition times which are the difference between presentation and decoder timestamps. – user1390208 Jan 03 '19 at 16:38
  • Thanks for the idea @user1390208. Can u suggest a standard muxing library that handles cts properly – Kiran Raj Jan 07 '19 at 06:32
  • @user1390208 by "unlike WebRTC", do you mean that WebRTC doesn't handle b frames? Does WebRTC support decoding raw h264 stream? – Zip Apr 26 '19 at 05:34
  • Correct, WebRTC does NOT handle b-frames. H264 without b-frames is supported. – user1390208 Apr 26 '19 at 16:28
  • i'am facing this problem did you find good solution – ashraf revo Jul 09 '19 at 16:53
  • Hey have you checked out [openH264](https://www.openh264.org/) and then implementing [webassembley](https://webassembly.org/) this could help with your problem – rawk Dec 10 '19 at 04:13
  • @KiranRaj have you tried https://github.com/samirkumardas/jmuxer it has demo's for you to mess with to get an understand of how to use it :D – Barkermn01 Dec 11 '19 at 12:58

1 Answers1

21

The best I have seen used (not had hands-on experience using it my self) is https://github.com/samirkumardas/jmuxer

There is an example of how to handle streaming data via WebSockets at https://github.com/samirkumardas/jmuxer/blob/master/example/index-h264.html

var socketURL = 'ws://localhost:8080';
var jmuxer = new JMuxer({
    node: 'player',
    mode: 'video',
    flushingTime: 1000,
    fps: 30,
    debug: true
});
var ws = new WebSocket(socketURL);
ws.binaryType = 'arraybuffer';
ws.addEventListener('message',function(event) {
     jmuxer.feed({
         video: new Uint8Array(event.data)
     });
});
ws.addEventListener('error', function(e) {
    console.log('Socket Error');
});
Barkermn01
  • 6,781
  • 33
  • 83