3

I am trying to implement perfect WebRTC negotiation for my small video conferencing application by considering the examples from the following page:

https://blog.mozilla.org/webrtc/perfect-negotiation-in-webrtc/

Unortunately I did not manage to make it fully work, especially mobile safari seems to handle rollback behavior its own way, here is the code that handles the rollback behavior:

      if (description) {
        const offerCollision = description.type == 'offer' && (makingOffer || pc.signalingState != 'stable');
        this.ignoreOffer = !this.polite && offerCollision;
        if (this.ignoreOffer) {
          return;
        }
        if (offerCollision) {
          await Promise.all([pc.setRemoteDescription({ type: 'rollback' }), pc.setRemoteDescription(description)]);

So when on mobile safari an offer collision (offerCollision === true) is detected and pc.setRemoteDescription({ type: 'rollback' }) is called as it's implemented in my code, it throws an error of type InvalidStateError. Taking a closer look at the documentation about this type of error in MDN (https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/setRemoteDescription#Browser_compatibility) shows:

"The RTCPeerConnection is closed, or it's in a state which isn't compatible with the specified description's type. For example, if the type is rollback and the signaling state is one of stable, have-local-pranswer, or have-remote-pranswer, this exception is thrown, because you can't roll back a connection that's either fully established or is in the final stage of becoming connected."

Checking the peer connections signaling state just before rolling back shows that it is in the state "have-local-offer" which should be ok since MDN says that rollback is not possible (throws InvalidStateError) in the states stable, have-local-pranswer, or have-remote-pranswer.

For the other case when my Desktop Chrome browser runs in an offer collision everything just works as expected with the same signaling state just before rollback is initiated.

Does someone here have an idea what's potentially wrong or to be handled differrently for mobile Safari.

gokumc
  • 138
  • 1
  • 9
  • https://github.com/smitkpatel16/rsvp_vidcall/blob/master/sources/js/chatroom.js Went through several iterations to get here – smitkpatel Jun 13 '20 at 06:12
  • Hey @smitkpatel, Thank you for your comment but I could not find any related code to rollback bahavior in the linked github repo. May you elaborate a bit more what you exactly mean or how your code can help me? – gokumc Jun 13 '20 at 12:47
  • Here is a demo that reproduces the error (occuring only on mobile Safari) https://jsfiddle.net/8eap2t9r/1/ – gokumc Jun 13 '20 at 13:59
  • Turns out to be a known bug in WebKit which was reported almost 3 years ago and still was not solved yet: https://bugs.webkit.org/show_bug.cgi?id=174656 I wonder if there is a workaround to "rollback" the peer connections state... One idea would be to initialize a new `RTCPeerConnection` instance. I am going to give it a try. – gokumc Jun 15 '20 at 14:22
  • Yes, that is what I was going to suggest. Close/Discard the older connection and create a new one. And did not encounter this error on my side since I was already doing that on reconnect. – smitkpatel Jun 17 '20 at 11:37
  • Did you find a nice solution? I’d be interested to see it :) – Dom Christie Feb 18 '21 at 15:10
  • Hey @DomChristie , Unfortunately no. Re-initialising the `RTCPeerConnection` did not work as a workaround, at least with a few trials from my side. Thus I did not implement perfect negotiation at all which means that my code is handling the negotiation process. It is a little bit annoying but luckily it needs to be implemented only once. – gokumc Feb 19 '21 at 11:25
  • 1
    Thanks! I persisted with discarding the the older connections and creating new ones. It required tearing down all event listeners, creating a new peer connection (+ event handlers), and signalling to the other client to do the same (https://github.com/domchristie/webrtc-hotwire-rails/blob/9050a9e6f279594ec9a176d59962effff406969b/app/assets/javascripts/models/webrtc_negotiation.js#L41). Once the other client receives the message to restart, it can create the offer to start the negotiation again. – Dom Christie Feb 21 '21 at 00:19
  • 1
    Your code looks promising to me. Thanks for sharing it! I am going to give it a try when I have some time left. I doubt it will be soon but I won't forget it (hanging a little bit on the automated app deployment). I also would like to mention that I like the way you're coding --> clear namings and obviously following the single responsibility principle. Feel free to post this as an answer so I can accept it in the meanwhile. – gokumc Feb 21 '21 at 12:45
  • Thanks for your comments—glad you like the demo! – Dom Christie Feb 25 '21 at 19:20

3 Answers3

3

As mentioned in the comments, Safari (both iOS/mobile and macOS) has a known bug with { type: 'rollback' }. It also does not yet support optional descriptions in setLocalDescription/setRemoteDescription, the latest perfect negotiation recommendation in the spec.

This can be fixed by discarding the colliding peer connections and retrying (as suggested). To do this, handle the error when setting a remote description with the following steps:

  1. Reset any state variables (e.g. makingOffer, isSettingRemoteAnswerPending)
  2. Close the peer connection by calling peerConnection.close()
  3. Tear down any peer connection event listeners (e.g. negotiationneeded, icecandidate, track)
  4. Signal to the other peer to do the same e.g. via websocket messaging
  5. On receiving the signal to restart, the other can go through the same steps, and finally create a new offer to kick off the negotiation process again.

An example of this flow can be seen in this demo.

Dom Christie
  • 4,152
  • 3
  • 23
  • 31
  • Doesn't this just start the whole process from beginning, potentially ending in the same situation? – Peter Knego Aug 06 '21 at 09:55
  • Yep, it's basically retry and hope to avoid a collision. Not ideal, but I'm not sure there are many other options :( – Dom Christie Aug 09 '21 at 00:49
  • It seems that Webkit updated libwebrtc to M92, which supports perfect negotiation: https://github.com/WebKit/WebKit/commit/5260c8ee94ae50bb686fd4e9a9349d69d0beefc8 , so we can probably expect this to land in Safari in the near future. – Peter Knego Aug 12 '21 at 07:27
2

Good news: Safari Tech Preview (15.4) supports implicit rollback in setRemoteDescription directly¹, which appears to also have fixed the explicit {type: 'rollback'} method you were using in the more backwards-compatible polyfill (though I've asked for clarification in the bug).

Bad news: At the time of writing, it's still not available in regular Safari (at 15.3 now) or Safari for iOS. But that should be only a matter of time.

Implicit rollback was designed specifically for perfect negotiation, letting you use the standard pattern directly. Here's a fiddle using it.

1. I've submitted a PR to update MDN to reflect this.

jib
  • 40,579
  • 17
  • 100
  • 158
  • Ok - so perfect negotiation is currently useless until the new release. The problem is that if you have 2 safari peers which have both sent an offer, there is no solution other than tear down and restart as neither can roll back, and you might end up in the same place again. I think your idea of out-of-band request to the impolite peer to renegotiate on behalf of the polite peer is the best solution I have seen so far. Just need to get it to work ... :P – FatalFlaw Jan 28 '22 at 12:58
  • 1
    ios 15.4 has been released and now it is supports explicit ```{type: 'rollback'}```! – AID Mar 15 '22 at 03:55
1

In case anyone finds themselves back here, I found that I could make perfect negotiation work with Safari if I make 2 changes:

  1. in onnegotiationneeded: if we are on safari, don't do the createOffer() or setLocalDescription() if the instance is polite.

  2. on incoming messages, in addition to the existing 'ignore offer' functionality, ignore type=='offer' messages if the instance is NOT polite and we are running on safari.

My assumption is that Safari webkit doesn't like being presented with offers if it's made an offer to the peer. I don't have anything to back that up with beyond the fact it seems to work - the errors disappear and I get video.

FatalFlaw
  • 1,077
  • 11
  • 19
  • 1
    Ignoring polite `negotiationneeded` requests avoids the error, but also leaves the polite peer with no way to negotiate new media, which only works if all changes are done from the impolite side. For this to be a solution, it would require signaling polite requests out of band to the impolite side to execute. – jib Jan 27 '22 at 21:56
  • That's a good point. If I'm right about safari webkit not allowing offers if it has an offer out, maybe one could allow the polite side to make offers once the initial connection has been established. I'll try it. – FatalFlaw Jan 28 '22 at 10:13
  • @FatalFlaw, any news? – AID Feb 24 '22 at 05:49