3

What I am trying to do

Stream live H.264 data over the network and use hardware acceleration to decode it as specified in the WWDC video so I can play it in my iOS app in real time.

What I have tried

I have read RFC 6184 extensively and believe I have my head wrapped around it. I assume the RTP packets I am receiving are in the format specified in the RFC. I am also assuming the stream will be in non-interleaved mode. So my application reads in each nalu packet and determines if it is a single nalu packet (type < 24), aggregated packet (type = 24), or fragmented packet (type = 28). From there I am just logging the data of what type of nalu I am receiving because I wanted to make sure the data is coming in the form I expect and being appropriately parsed before moving on.

What happens

When I log the nalu types I get nal units with a type of 0, 1, 2 with occasional PPS nalus as shown in the picture.

Log of NAL unit packet types received I know that I should receive a SPS and a PPS as the first 2 packets so I know something is wrong.

My question

I am using VLC to stream the data. I am transcoding it in H.264 and with the encapsulation format MPEG TS. Now, when I first started this I thought I as long as I was streaming H.264 it would be in the format specified in the RFC. But after doing some research about MPEG TS and how it is a container format I think my stream is not in the format specified in the RFC.

If I am actually streaming the video in a format different from the RFC, how would I use VLC (or an equivalent program) to stream the data in the form of the RFC? All the posts on SO such as this and the WWDC video talk about looking for a start code (0x0001 or 0x000001) for each nalu but the RFC never mentions it. So I am not sure if that is the reason I am getting weird results is due to me not looking for a start code or if it is because I am not receiving the data in the RFC format. I greatly appreciate any help especially since I am new to video decoding in general and feel like I am almost there!

Community
  • 1
  • 1
ddelnano
  • 459
  • 5
  • 19
  • Have you found a solution yt? – Jack Song Jun 08 '16 at 01:54
  • No this project I had to reduce the scope to exclude a live video stream so unfortunately it never was implemented. Are you working on something similar to this? – ddelnano Jun 08 '16 at 01:55
  • Yeah...sadly not much source can be found.. – Jack Song Jun 08 '16 at 01:57
  • I would try this book http://files.cnblogs.com/files/irish/The_H.264_advanced_video_compression_standard.pdf . I never got the chance to read it, but it looked promising. The other thing I was going to try which I never got the chance to was to read the source of ffmpeg or another opensource tool that can do the video streaming / compression / whatever. I don't know for sure that it will help but searching around got me almost no where. – ddelnano Jun 08 '16 at 02:02
  • @JackSong any luck with that? – ddelnano Jan 05 '17 at 04:51
  • Hey! Thx 4 checking back, but no...we've dropped the project since.. – Jack Song Jan 14 '17 at 04:36

0 Answers0