1

I looked in The RFC and noting could explain why the following happens(Though the decoder can still produce the original movie).

I transmitted the H.264/AVC nals using VSS h.264 encoder, the byte stream looked something like this E5 46 0E 4F FF A0 23...

when I read the movie data one the receiver side after the RTP Broadcaster/RTSP receiver, I get extra unknown data but always in the same places, 8 bytes are added before Start Code prefix (0x00000001), and 2 bytes are added after Start Code prefix it looks something like this.

XX XX XX XX XX XX XX XX 00 00 00 01 XX XX, then I look in the Wireshark and i could see that the RTP adds the bytes to the data payload.

Why does it happens why? and why the decoder seems to cope well with those extra bytes?!

Canttouchit
  • 3,149
  • 6
  • 38
  • 51

2 Answers2

1

Thats some messed up stream... And you can mess it up even more, and it will still work, because decoder parses it for 0x000001 start code, skipping the bytes that are added at the beginning. Those two new bytes at the end must be H264 fragmentation bytes... or something H264 related since they work.

So basically, this is due to defective packetizer/RTSP source filter. My guess is that if you ASCII encode those 8 bytes you will get the vendor name of the RTSP source filter... xD

Cipi
  • 11,055
  • 9
  • 47
  • 60
  • So you are saying that after 0x000001 my H.264 decoder skips the the added two bytes(00 00 00 01 XX XX)?! That's seems little weird since the decoder expect the header to come after each 0x000001. – Canttouchit Oct 06 '11 at 18:10
  • Well if there is packetization, or aggregation involved you would have 2 extra bytes there. So, I'm saying that 2 bytes that are extra, are there for some purpose. You can check the payload format for H264 transmitted over RTP if you haven't already (http://www.ietf.org/rfc/rfc3984.txt) and see what those two bytes are. Also, post their values here, maybe I can help... ;) – Cipi Oct 06 '11 at 21:20
0

As I mentioned in another post Changing NALU h.264/avc, for RTP encupsulation, H.264 is transmitted over RTP as defined in RFC 3984. This in particular defines how exactly large NAL units are broken into smaller parts that fit smaller message sizes, suchas UDP datagram size. That is, fragmentation.

Receiver depacketizes the data and restores NALUs, and it uses this extra information to do the job.

So what you essentially need is to compare raw data you have against RFC 3984 format. Also, Wireshark already does partially this for you by dissecting traffic into readable items.

Community
  • 1
  • 1
Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • Still it does not answer my question, the extra information is not completely removed, after the receiver used it its propagate to the decoder the xx represent the extra information the receiver does not removes from the nals "XX XX XX XX XX XX XX XX 00 00 00 01 XX XX" – Canttouchit Sep 28 '11 at 09:10
  • Well, first of all start codes do not have to be transmitted over RTP. This is why you would still have to go through RFC and compare against your stream. You still see start codes? Your device might be sending them for some reason, maybe due to a bug. Check this out, this is a start of SEI NAL RTP packet, no start codes and it's a valid stream. http://img691.imageshack.us/img691/9823/image003t.png – Roman R. Sep 28 '11 at 10:02
  • I meant AFTER the RTSP receiver I sample the data, why this data stays if the h.264 encoder does not need it?! – Canttouchit Sep 28 '11 at 10:06
  • I am afraid I still miss your point. Your question is about RTSP/RTP, RTP carrier does not use start codes. RTSP/RTP client may use start codes in order to reconstruct H.264 stream, or may choose to not use it (e.g. in Windows, DirectShow AVC1 media type is H.264 video without start codes). If you see the codes in RTP, then sending party for some reason includes them. If they are not on wire, and RTSP receiver adds them then it has nothing to do with RTP. – Roman R. Sep 28 '11 at 10:16
  • H.264 encoder generates nals, RTP broadcaster takes NALs sometimes whole sometimes segments them. when I look in the WireShark in the RTP Data payload I see NAL's data (Sometimes whole sometimes segmented) but in the RTP data payload there is additional data that I didnt expected RTP to add, I thought it negotiate with the RTSP with the RTP header's fields, but whatever, I kept tracking the Nals data as it progressed through the system and noticed that the additional Data added to the NAL is not removed, after decupsulation of each RTP header using RTSP... – Canttouchit Sep 28 '11 at 11:35
  • the additional data I was tracking with WireShark stays? and I think the RTSp adds more data, to be more accurete, each RTP's packet Data is refered by RTSP receiver as NAL's data and the RTSP simply adds start code prefix 0x00000001 between RTP packets data, however... – Canttouchit Sep 28 '11 at 11:37
  • The 2 bytes I did track with Wireshark stays in and refered as Nal's data something like this 0x00000001XXXX, and RTSP adds some more data 8 bytes to be accurate before the 0x00000001, those Bytes are added by RTSP receiver since I didn't see the Bytes in the RTP packets captured by Wireshark, now my question what are those bytes why they remain after all the decupsulation possess is completed? – Canttouchit Sep 28 '11 at 11:41
  • My understanding - without having full data log in front of me - is that receiver does not care about this extra data and just unwraps and defragments the payload. If sender included that extra data, it will expectedly be left in the data stream, why would receiver choose to drop it? Then later decoder might be just ignoring the data (esp. if is following the real H.264 NAL), or it might be successfully skipping it, or it's modified H.264 stream where custom receiver would extract this additional data before actually decoding video. – Roman R. Sep 28 '11 at 11:48