2

I am searching for low level implementation details on muxing RTP and RTCP streams using BUNDLE on a Java based server. With Chrome as my source, this is what a local SDP looks like:

o=- 8554465656018336221 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video data
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 111 103 104 0 8 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:Vw+winZTN4ejhvQJ
a=ice-pwd:ufBTUw/iszvCbL53dmPHQAYK
a=ice-options:google-ice
a=fingerprint:sha-256 5C:C6:19:38:4D:54:57:71:16:3F:67:A6:C8:21:CC:29:88:85:22:86:53:E5:7B:3F:3D:A4:5C:E5:BC:29:D8:B5
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:126 telephone-event/8000
a=maxptime:60
m=video 1 RTP/SAVPF 100 116 117
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:Vw+winZTN4ejhvQJ
a=ice-pwd:ufBTUw/iszvCbL53dmPHQAYK
a=ice-options:google-ice
a=fingerprint:sha-256 5C:C6:19:38:4D:54:57:71:16:3F:67:A6:C8:21:CC:29:88:85:22:86:53:E5:7B:3F:3D:A4:5C:E5:BC:29:D8:B5
a=setup:actpass
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=recvonly
a=rtcp-mux
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
m=application 1 DTLS/SCTP 5000
c=IN IP4 0.0.0.0
a=ice-ufrag:Vw+winZTN4ejhvQJ
a=ice-pwd:ufBTUw/iszvCbL53dmPHQAYK
a=ice-options:google-ice
a=fingerprint:sha-256 5C:C6:19:38:4D:54:57:71:16:3F:67:A6:C8:21:CC:29:88:85:22:86:53:E5:7B:3F:3D:A4:5C:E5:BC:29:D8:B5
a=setup:actpass
a=mid:data
a=sctpmap:5000 webrtc-datachannel 1024

I've google'd etc and have not found what I need as of yet. I did find this page and it only has mostly high-level info, but again I need more: https://datatracker.ietf.org/doc/html/draft-ejzak-avtcore-rtp-subsessions-01

In addition, I am subscribed to https://groups.google.com/forum/#!aboutgroup/discuss-webrtc but I haven't seen any low level information about how muxing works with 

a=group:BUNDLE audio video data
is used.

Related questions:
WebRTC java server trouble
How can I mux/demux RTP media from one stream?

Community
  • 1
  • 1
Paul Gregoire
  • 9,715
  • 11
  • 67
  • 131

1 Answers1

2

All this is meaning that the data is being sent over the same port. This does not mean that the packets themselves are modified in any way.

The way to separate out the packets(knowing which is audio/video and their respective control packets) are to check them against their respective SSRC in the RTP/RTCP packet headers. This way you don't modify your video stream given the audio control packet and vice versa.

In chrome, you can make it alert to the respective SSRC ids through the SDP exchange by including a=ssrc:<ID> for each media level(one for video and one for audio).

It also looks like your SDP is set to recvonly for both media types. This means that it does not RECEIVE any RTCP and will only send them back to the sender so that the streams can be modified accordingly.

Benjamin Trent
  • 7,378
  • 3
  • 31
  • 41
  • So the RTP packet bytes are sent without any additional information or headers when bundled? Lastly, I don't see an a:ssrc lines? – Paul Gregoire Jul 10 '14 at 15:08
  • the lines are not there as this is `recvonly`...at least that is what I assume(all work I have done is `sendrecv` and the ssrc lines were present). The RTP packet bytes are sent and each header contains its unique `SSRC` ID from what I have seen in practice and parsing these myself. So, when they are bundled, it just signifies over the same port, not that the packets are merged in some way. Technically, it would be impossible to merge them as they would have different ClockRates. – Benjamin Trent Jul 10 '14 at 15:22
  • I would assume that chrome doesn't use sendrecv because this is a live broadcast situation. – Paul Gregoire Jul 10 '14 at 15:24
  • You could also make educated "guesses" given the rtp payload and the packet contents as to media type and then match it up against its included SSRC in the header for the rest of the session. – Benjamin Trent Jul 10 '14 at 15:26
  • Yes, the reason the SSRC attributes are not there are because it is `recvonly`. The SDP that IT receives should indicate the SSRC IDs that are being sent to it from the server. Google will not know the SSRC IDs unless provided them as it is not generating the media. – Benjamin Trent Jul 10 '14 at 18:53