I'm going to just list everything I know since this could all be important.
I bought a simple H264 streaming player from Amazon to test. Specifically this one: http://www.amazon.com/OPR-NH100-Encoder-Broadcast-Recording-replace/dp/B00NIFJYEC/
The settings are as follows:
As you can see I'm using the Main Profile for H264. In Apache2 I'm proxying the connection through localhost. So on Ubuntu in /etc/apache2/sites-enabled/000-default.conf I have:
<VirtualHost *:80>
<Proxy "*">
Allow from all
</Proxy>
ProxyPass /hdmi http://192.168.1.168/hdmi retry=0
ProxyPassReverse /hdmi http://192.168.1.168/hdmi
</VirtualHost>
This is fairly basic. Essentially this bypasses any cross-domain issues since the stream is local.
At this point if I open http://127.0.0.1/hdmi with VLC Media Player's network stream viewer I can see the stream running just fine. (I can play a DVD or stream Cable which both work fine).
So then I tried to use this example here for Media Source Extensions. It works fine with the mp4 file so I modified it to say:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
</head>
<body>
<video controls></video>
<script>
var video = document.querySelector('video');
var assetURL = 'hdmi';
// Need to be specific for Blink regarding codecs
// ./mp4info frag_bunny.mp4 | grep Codec
var mimeCodec = 'video/mp4; codecs="avc1.4D402A, mp4a.40.2"';
if ('MediaSource' in window && MediaSource.isTypeSupported(mimeCodec)) {
var mediaSource = new MediaSource;
//console.log(mediaSource.readyState); // closed
video.src = URL.createObjectURL(mediaSource);
mediaSource.addEventListener('sourceopen', sourceOpen);
} else {
console.error('Unsupported MIME type or codec: ', mimeCodec);
}
function sourceOpen (_) {
//console.log(this.readyState); // open
var mediaSource = this;
var sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
fetchAB(assetURL, function (buf) {
sourceBuffer.addEventListener('updateend', function (_) {
mediaSource.endOfStream();
video.play();
//console.log(mediaSource.readyState); // ended
});
sourceBuffer.appendBuffer(buf);
});
};
function fetchAB (url, cb) {
console.log(url);
var xhr = new XMLHttpRequest;
xhr.open('get', url);
xhr.responseType = 'arraybuffer';
xhr.onload = function () {
cb(xhr.response);
};
xhr.send();
};
</script>
</body>
</html>
In Firefox it accepts the codec information as valid and makes a successful network connection to the hdmi stream. The issue is it doesn't play. It just spins when I press play with no error.
I thought maybe the codec was wrong. From the above code I'm using:
video/mp4; codecs="avc1.4D402A, mp4a.40.2"
I decided to check to make sure this was right so I ran:
ffprobe -show_streams http://127.0.0.1/hdmi
Input #0, mpegts, from 'http://127.0.0.1/hdmi':
Duration: N/A, start: 1780.838256, bitrate: 130 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0.0[0x100]: Video: h264 (Main), yuv420p, 1920x1080, 90k tbr, 90k tbn, 180k tbc <--- Main Profile so 4D
Stream #0.1[0x101]: Audio: aac, 48000 Hz, stereo, s16, 130 kb/s
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 <--- AVC so avc1
codec_type=video
codec_time_base=1/180000
codec_tag_string=[27][0][0][0]
codec_tag=0x001b
width=1920
height=1080
has_b_frames=0
pix_fmt=yuv420p
level=42 <-- Level is 42 which is 0x2A
id=0x100
r_frame_rate=180000/2
avg_frame_rate=0/0
time_base=1/90000
start_time=1780.911511
duration=N/A
[/STREAM]
[STREAM]
index=1
codec_name=aac <---- AAC = mp4a
codec_long_name=Advanced Audio Coding
codec_type=audio
codec_time_base=1/48000
codec_tag_string=[15][0][0][0]
codec_tag=0x000f
sample_rate=48000.000000
channels=2
bits_per_sample=0
id=0x101
r_frame_rate=0/0
avg_frame_rate=375/8
time_base=1/90000
start_time=1780.838256
duration=N/A
[/STREAM]
This clearly shows Main Profile being used just like I set in the streamer's configuration settings. It also shows Level 42 being used which I believe translates to avc1.4D402A since all the Main Profile examples I've seen use 4D40 then the 2A is 42 in hex. Am I reading this wrong? (I've tried tons of other combinations with no change).
The audio is AAC so mp4a is correct, but I'm not confident about the 40.2 part. I read a few examples online saying to use mp4file so I used VLC to save the raw stream then ran:
mp4file --dump networkhdmi.mp4
But that fails since the stream doesn't start with a valid non-delta frame since it's a stream. Here's the raw stream file if you need to test and see the problem:
http://sirisian.com/randomfiles/networkhdmi.mp4
I even tried to run the file with that raw file inside of the browser instead of using the stream and it failed.
I'm starting to think that FireFox's decoder for MP4 doesn't work if the first frame is a delta frame. ffprobe shows when it analyzes the stream:
non-existing PPS referenced
non-existing PPS 1 referenced
decode_slice_header error
no frame!
Could this be tripping the browser up? Seems like VLC isn't bothered by it.
I think that networkhdmi.mp4 file is a really good testcase since it's exactly what's coming from the stream and shows the problem perfectly. In the code above just change:
var assetURL = 'hdmi';
to:
var assetURL = 'networkhdmi.mp4';
Here's an example on my site:
http://sirisian.com/randomfiles/networkhdmi.html
If you need any other information just ask in a comment and I'll supply it. I've been working on this for two days with no progress. (Also in case anyone asks I've tried using the H264 High Profile and it doesn't work either in the browser. Also tried variable bitrate and tried both Chrome and Firefox latest versions).
One of my friends commented:
the browser cannot process any streams with MSE. It requires completely independent MP4 chunks like what you would get from a MPEG-DASH stream or similar. MP4 chunk should start with AAVCS or whatever extradata, then a IDR keyframe and then delta frames
There are javascript libraries that transmux mpeg2-ts to mp4 so it can be played by MSE; however, they are all based on HLS where the .ts files are separate chunks, and it transmuxes one when the download is done what you will have to do is detect IDR yourself, and stop and transmux everything you have so far and feed that to MSE, continuously. none of them support progressive http download of TS
This seems right? I tried to use the HLS which doesn't work with this stream. My friend mentioned:
HLS is defined as a m3u8 which has relative URLs to TS fragemnts while you have a CONTINUOUS TS since MSE accepts only MP4 and MP4 is not streamable
This sounds like a lot of work to make this function. Since I have a proxy I guess I can try to use ffmpeg with nginx-rtmp or something. I'm gonna go investigate and test.
