0

I've written an Android app to stream video from my nest cameras and doorbell. I have a valid access token and can retrieve a list of all my nest devices. However, when I send a GenerateWebRtcStream to any device, it returns the same 400 error.

I have verified that my offer:

  1. Has a=recvonly, and that it also has H264 as an available video codec. As indicated in documentation
  2. Has a data channel (as suggested by another solution to this problem).
  3. Has the audio stream data before the video stream data (as suggested by a second solution to this problem).
  4. [Updated] Have created the offer with a PeerConnection initialized with SdpSemantics.UNIFIED_PLAN

Additional information:

My Android app is based (almost entirely) off this very helpful blog. The app works when connecting to a non-nest camera (i.e. instance of itself on another device), so the offer and logic is valid on some level.

There appears to be very little information on what offer the Nest Camera's will accept, so I'm posting the generated offer and command I'm posting to the API in the hopes that someone has an idea on what I might need to change.

Many thanks in advance.

sdpOffer:

v=0
o=- 5150109222623794114 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1 2
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:KQKv
a=ice-pwd:QvV7coYbshMtmKmkqpBno0xz
a=ice-options:trickle renomination
a=fingerprint:sha-256 C2:16:13:A3:80:1A:8E:96:21:98:4B:8C:03:26:93:DE:FA:88:C5:8B:DB:06:62:87:FF:BA:D4:0C:98:2A:3E:C9
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 125 100 101 127
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:KQKv
a=ice-pwd:QvV7coYbshMtmKmkqpBno0xz
a=ice-options:trickle renomination
a=fingerprint:sha-256 C2:16:13:A3:80:1A:8E:96:21:98:4B:8C:03:26:93:DE:FA:88:C5:8B:DB:06:62:87:FF:BA:D4:0C:98:2A:3E:C9
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:100 red/90000
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:127 ulpfec/90000
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:KQKv
a=ice-pwd:QvV7coYbshMtmKmkqpBno0xz
a=ice-options:trickle renomination
a=fingerprint:sha-256 C2:16:13:A3:80:1A:8E:96:21:98:4B:8C:03:26:93:DE:FA:88:C5:8B:DB:06:62:87:FF:BA:D4:0C:98:2A:3E:C9
a=setup:actpass
a=mid:2
a=sctp-port:5000
a=max-message-size:262144

POST Body:

{
   {
   "command":"sdm.devices.commands.CameraLiveStream.GenerateWebRtcStream",
   "params":{
      "sdpOffer":"v=0\r\no=- 5150109222623794114 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1 2\r\na=msid-semantic: WMS\r\nm=audio 9 UDP\/TLS\/RTP\/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:KQKv\r\na=ice-pwd:QvV7coYbshMtmKmkqpBno0xz\r\na=ice-options:trickle renomination\r\na=fingerprint:sha-256 C2:16:13:A3:80:1A:8E:96:21:98:4B:8C:03:26:93:DE:FA:88:C5:8B:DB:06:62:87:FF:BA:D4:0C:98:2A:3E:C9\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=extmap:3 http:\/\/www.ietf.org\/id\/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus\/48000\/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC\/16000\r\na=rtpmap:104 ISAC\/32000\r\na=rtpmap:9 G722\/8000\r\na=rtpmap:102 ILBC\/8000\r\na=rtpmap:0 PCMU\/8000\r\na=rtpmap:8 PCMA\/8000\r\na=rtpmap:106 CN\/32000\r\na=rtpmap:105 CN\/16000\r\na=rtpmap:13 CN\/8000\r\na=rtpmap:110 telephone-event\/48000\r\na=rtpmap:112 telephone-event\/32000\r\na=rtpmap:113 telephone-event\/16000\r\na=rtpmap:126 telephone-event\/8000\r\nm=video 9 UDP\/TLS\/RTP\/SAVPF 96 97 98 99 125 100 101 127\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:KQKv\r\na=ice-pwd:QvV7coYbshMtmKmkqpBno0xz\r\na=ice-options:trickle renomination\r\na=fingerprint:sha-256 C2:16:13:A3:80:1A:8E:96:21:98:4B:8C:03:26:93:DE:FA:88:C5:8B:DB:06:62:87:FF:BA:D4:0C:98:2A:3E:C9\r\na=setup:actpass\r\na=mid:1\r\na=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http:\/\/www.ietf.org\/id\/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:12 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/playout-delay\r\na=extmap:11 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/video-content-type\r\na=extmap:7 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/video-timing\r\na=extmap:8 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/color-space\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8\/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx\/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9\/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=rtpmap:99 rtx\/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:125 H264\/90000\r\na=rtcp-fb:125 goog-remb\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 ccm fir\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:100 red\/90000\r\na=rtpmap:101 rtx\/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:127 ulpfec\/90000\r\nm=application 9 UDP\/DTLS\/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:KQKv\r\na=ice-pwd:QvV7coYbshMtmKmkqpBno0xz\r\na=ice-options:trickle renomination\r\na=fingerprint:sha-256 C2:16:13:A3:80:1A:8E:96:21:98:4B:8C:03:26:93:DE:FA:88:C5:8B:DB:06:62:87:FF:BA:D4:0C:98:2A:3E:C9\r\na=setup:actpass\r\na=mid:2\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n"
   }
}

Error Response:

{
   "error":{
      "code":400,
      "message":"sdp_offer contains an invalid value.",
      "status":"INVALID_ARGUMENT"
   }
}
jeffrwatts
  • 11
  • 3
  • Are you using the Unified format or Plan B format for your SDP? I believe only Unified is supported. – Gothic Jan 10 '22 at 23:05
  • Thank you @BTGunner If only unified is supported, then I think this might explain the failure. I'm using the Android API in the WebRTC library. After digging through the source, it appears that [PeerConnection.java](https://webrtc.googlesource.com/src/+/refs/heads/main/sdk/android/api/org/webrtc/PeerConnection.java) is initialized to SdpSemantics.PLAN_B (line 614). I unfortunately don't see a way to construct PeerConnection with UNIFIED_PLAN. Really appreciate your help. I have a path forward to try a few more things. – jeffrwatts Jan 12 '22 at 01:41
  • Found out how to enable SdpSemantics.UNIFIED_PLAN when creating PeerConnection, but still unfortunately receive the same failure. Updated sdpOffer and POST Body with the Unified Plan offer. – jeffrwatts Jan 14 '22 at 06:06
  • can you try using `a=sendrecv` in the audio section instead of `a=recvonly`? I know the docs say to use `a=recvonly` but maybe see if it works. Also, if I may ask, what device model are you trying to stream from? – Gothic Jan 27 '22 at 05:07
  • I just tried `a=sendrecv` for audio (by adding an audio track prior to generating the offer). Unfortunately I get the same response error response. I'm using a Pixel 6. Thanks again for the continued help. – jeffrwatts Jan 27 '22 at 15:14
  • Sorry, by device model I meant what Nest camera model. Also, there is a public web sample available that handles WebRTC live streaming, have you tried using that to see if it works for your device? https://github.com/google/device-access-sample-web-app (referenced from https://developers.google.com/nest/device-access/samples/web-app) – Gothic Feb 01 '22 at 19:35
  • I have a Nest Camera (battery) - `nq-user 1.59 OPENMASTER 279935` and Nest Doorbell (battery) `gp-user 1.59 OPENMASTER 278788`. I'll have a look at the public web sample app, and see what I can learn. Thanks again – jeffrwatts Feb 03 '22 at 14:57

1 Answers1

1

Got the sample app at github.com/google/device-access-sample-web-app running as suggested by @BTGunner (thanks again!!!), and got to see what a successful request looks like.

The offer that I'm generating is accepted. The problem was that the documentation and implementation have deviated for GenerateWebRtcStream. The current documentation indicates that the offer description should be named offerSdp (which fails). The sample app instead uses offer_sdp and this returns a valid answer.

jeffrwatts
  • 11
  • 3
  • Glad you got it to work! Something I only just noticed in your original question though... your request payload actually has "sdpOffer" as the field name, when the docs have "offerSdp". Can you try it again with offerSdp as the field name and see if that works too? – Gothic Feb 04 '22 at 18:12
  • . Yes, offerSdp works... I can't believe that was it. At least I have had a nice education on SDP formats pouring through comparisons :). Appreciate your help again @BTGunner. Best. – jeffrwatts Feb 05 '22 at 00:14