2

Does anyone know how to extract image frames from a SDP video output? I'm using a Nest battery camera. The wired version gave me an RTSP stream which was easy to extract frames. However, the battery version gave me a SDP output which is hard to make sense of. I've looked at a few posts on stackoverflow but none seemed too promising:

How to use the answerSDP returned from sdm.devices.commands.CameraLiveStream.GenerateWebRtcStream to establish a stream with google nest cam

Executing FFmpeg recording using in-line SDP

Even being able to stream SDP to a mp4 file using ffplay would be a nice start. But ultimately I would like to run a python script to extract frames from SDP output.

I must admit, SDP (session description protocol) seems pretty long and complicated compared to working with RTSP streams. Anyway to simply convert an SDP stream to a RTSP stream?

https://andrewjprokop.wordpress.com/2013/09/30/understanding-session-description-protocol-sdp/

Thanks! Jacob

SDP output looks something like this:

v=0\r\no=- 0 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 2 1\r\na=msid-semantic: WMS 16733765853514488918/633697675 virtual-6666\r\na=ice-lite\r\nm=audio 19305 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 142.250.9.127\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate: 1 udp 2113939711 2607:f8b0:4002:c11::7f 19305 typ host generation 0\r\na=candidate: 1 tcp 2113939710 2607:f8b0:4002:c11::7f 19305 typ host tcptype passive generation 0\r\na=candidate: 1 ssltcp 2113939709 2607:f8b0:4002:c11::7f 443 typ host generation 0\r\na=candidate: 1 udp 2113932031 142.250.9.127 19305 typ host generation 0\r\na=candidate: 1 tcp 2113932030 142.250.9.127 19305 typ host tcptype passive generation 0\r\na=candidate: 1 ssltcp 2113932029 142.250.9.127 443 typ host generation 0\r\na=ice-ufrag:UVDO0GOJASABT95E\r\na=ice-pwd:FRILJDCJZCH+51YNWDGZIN0K\r\na=fingerprint:sha-256 24:53:14:34:59:50:89:52:72:58:04:57:71:BB:C4:89:91:3A:52:EF:C0:5A:A5:EC:B5:51:64:80:AC:13:89:8A\r\na=setup:passive\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendrecv\r\na=msid:virtual-6666 virtual-6666\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=ssrc:6666 cname:6666\r\nm=video 9 UDP/TLS/RTP/SAVPF 108 109\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:UVDO0GOJASABT95E\r\na=ice-pwd:FRILJDCJZCH+51YNWDGZIN0K\r\na=fingerprint:sha-256 24:53:14:34:59:50:89:52:72:58:04:57:71:BB:C4:89:91:3A:52:EF:C0:5A:A5:EC:B5:51:64:80:AC:13:89:8A\r\na=setup:passive\r\na=mid:1\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendrecv\r\na=msid:16733765853514488918/633697675 16733765853514488918/633697675\r\na=rtcp-mux\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=rtcp-fb:108 goog-remb\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=ssrc-group:FID 633697675 3798748564\r\na=ssrc:633697675 cname:633697675\r\na=ssrc:3798748564 cname:633697675\r\nm=application 9 DTLS/SCTP 5000\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:UVDO0GOJASABT95E\r\na=ice-pwd:FRILJDCJZCH+51YNWDGZIN0K\r\na=fingerprint:sha-256 24:53:14:34:59:50:89:52:72:58:04:57:71:BB:C4:89:91:3A:52:EF:C0:5A:A5:EC:B5:51:64:80:AC:13:89:8A\r\na=setup:passive\r\na=mid:2\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n

user1446797
  • 129
  • 2
  • 9
  • Here are a couple more posts I saw that just weren't simple: https://stackoverflow.com/questions/45907538/joining-a-video-stream-in-ffmpeg-by-passing-an-sdp-file-as-inline-data https://stackoverflow.com/questions/70943901/webrtc-answer-sdp-returns-recvonly-instead-of-sendrecv. Anyone have a simple way to record as mp4 and/or extract frames from SDP video output? – user1446797 Jul 24 '22 at 10:16
  • Couple other ffmpeg options that didn’t work or I don't understand enough: https://ffmpeg.org/ffmpeg-all.html 
 • i.) ffmpeg -rtsp_flags listen -i rtsp://ownaddress/live.sdp output • 

ii.) ffplay -f lavfi "movie=filename='1.sdp':format_opts='protocol_whitelist=file,rtp,udp\:protocol_blacklist=http'" – user1446797 Jul 25 '22 at 14:28

2 Answers2

0

The sdp needs to be passed to your webrtc library which will initiate the connection and begin the video stream. Webrtc is built into browsers but if you're building an app on a different platform you'll need a compatible webrtc library.

Ethan
  • 136
  • 4
0

SDP is used by webrtc to establish a P2P connection. Since WebRTC is part of google's chromium I guess you have to establish a p2p connection using WebRTC with that camera then get YUV images from the video sink that you will attach to the connection.

Stoica Mircea
  • 782
  • 10
  • 22