Unsupported IOSurface format: 0x26424741 using twilio video in scenekit

Question

I am using twilio to send a video and use that video in a scenekit as a texture. But the problem is it works fine with iPhone X, but it gave this error Unsupported IOSurface format: 0x26424741 on iPhone XR and XS.

this is what I am doing:

Get Video:

func subscribed(to videoTrack: TVIRemoteVideoTrack, publication: TVIRemoteVideoTrackPublication, for participant: TVIRemoteParticipant) {
    print("Participant \(participant.identity) added a video track.")
    let remoteView = TVIVideoView.init(frame: UIWindow().frame,
                                       delegate:self)
    videoTrack.addRenderer(remoteView!)
    delegate.participantAdded(with: remoteView!)
}

delegate:

func participantAdded(with videoView: UIView) {
    sceneView.addVideo(with: videoView)
}

and add video to plane:

func addVideo(with view: UIView){
    videoPlane.geometry?.firstMaterial?.diffuse.contents = view
}

score 1 · Accepted Answer · answered Jul 18 '19 at 09:04

The problem was actually with renderingType of remoteView. For older devices using metal was fine but newer devices it needed openGLES. I dont know why but it was the fix.

I used this solution to find out the device type.

Next I determined which renderingType to use

var renderingType: VideoView.RenderingType {
    get{
        let device = UIDevice()
        switch device.type{
        case .iPhoneXS:
            return .openGLES
        case .iPhoneXR:
            return .openGLES
        case .iPhoneXSMax:
            return .openGLES
        default:
            return .metal
        }
    }
}

And used it to initialize remoteView

func didSubscribeToVideoTrack(videoTrack: RemoteVideoTrack, publication: RemoteVideoTrackPublication, participant: RemoteParticipant) {
    print("Participant \(participant.identity) added a video track.")
    let remoteView = VideoView.init(frame: UIWindow().frame,
                                    delegate:self,
                                    renderingType: renderingType)
    videoTrack.addRenderer(remoteView!)
    delegate.participantAddedVideo(for: participant.identity, with: remoteView!)
}

Unsupported IOSurface format: 0x26424741 using twilio video in scenekit

1 Answers1