2

I am able to send Replaykit video samples to the remote peer but with no Audio. What to use instead of RTCVideoFrame? Is there a way I can send a Audio Video data together?

Below is the snippet:

 var peerConnectionFactory: RTCPeerConnectionFactory?
        var localVideoSource: RTCVideoSource?
        var videoCapturer: RTCVideoCapturer?
        func setupVideoCapturer(){
                // localVideoSource and videoCapturer will use 
                localVideoSource = self.peerConnectionFactory!.videoSource() 
                videoCapturer = RTCVideoCapturer()
                localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

                let videoTrack : RTCVideoTrack =   self.peerConnectionFactory!.videoTrack(with: localVideoSource, trackId: "100”)

                let mediaStream: RTCMediaStream = (self.peerConnectionFactory?.mediaStream(withStreamId: “1"))!
                mediaStream.addVideoTrack(videoTrack)
                self.newPeerConnection!.add(mediaStream)
            }


     override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
         switch sampleBufferType {
                   case RPSampleBufferType.video:

                // create the CVPixelBuffer
                let pixelBuffer:CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!;

                // create the RTCVideoFrame
                var videoFrame:RTCVideoFrame?;
                let timestamp = NSDate().timeIntervalSince1970 * 1000
                videoFrame = RTCVideoFrame(pixelBuffer: pixelBuffer, rotation: RTCVideoRotation._0, timeStampNs: Int64(timestamp))
                // connect the video frames to the WebRTC
                localVideoSource.capturer(videoCapturer, didCapture: videoFrame!)

                break
            }
        } 
user2801184
  • 329
  • 7
  • 27
  • Hi @user2801184, Did you manage to receive video streaming using the code above? In my case I could not :( – Aboulfouz Feb 14 '19 at 10:26
  • Hi @Aboulfouz I just used above code for sending the RTCVideoFrame to another peer. Please refer to apprtc latest demo and use broadcast app like mobcrush. Not sure what is wrong with your code. can you post the code? – user2801184 Feb 15 '19 at 16:52
  • Hi @user2801184 The problem was with the timestamp. When I changed the timestamp to let ctm = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) ctm.value. It worked but the video freezes after 10 secs :( – Aboulfouz Feb 20 '19 at 14:52
  • @Aboulfouz: yes I encounter same issue where video freezes at specific sample using the ipad – user2801184 Feb 21 '19 at 01:31
  • can you please share how did you solve it? – Aboulfouz Feb 21 '19 at 06:34
  • I am still encountering same issue. can you ask your question here https://stackoverflow.com/questions/50267456/using-webrtc-to-send-an-ios-devices-screen-capture-using-replaykit/50953458?noredirect=1#comment96031230_50953458 – user2801184 Feb 21 '19 at 16:36
  • I solved the issue by updating to the newest GoogleWebRTC library and use the following command: localVideoSource?.adaptOutputFormat(toWidth: 768, height: 1024, fps: 10) – Aboulfouz Feb 25 '19 at 16:28
  • @Aboulfouz: Can you share the latest webrtc framework files? what version of webrtc libraries are you using? – user2801184 Feb 26 '19 at 17:24
  • GoogleWebRTC (1.1.26791), You can use Pod installation to get this version – Aboulfouz Feb 28 '19 at 13:46
  • Thanks, will try that! – user2801184 Feb 28 '19 at 20:09
  • That is not working anymore :( – decades Apr 16 '19 at 22:03
  • @user2801184 Did you have any solution for audio?? Please share with me. – Parth Patel Sep 06 '19 at 13:38
  • @ParthPatel: Sorry, I don't think there is solution for audio. Did you get video working? Can you share the solution? – user2801184 Sep 06 '19 at 23:07
  • @user2801184 For me, Video share is working now for me. I used your code for this. – Parth Patel Sep 07 '19 at 05:11
  • @ParthPatel: Can you please share the libWebRTC-arm64 – user2801184 Mar 11 '20 at 21:00
  • @user2801184 Sorry, I can't share lib file it is not with me but I can give you how you can do this. – Parth Patel Mar 12 '20 at 07:14
  • @user2801184 You can download opensource code from here chromium.googlesource.com/external/webrtc – Parth Patel Mar 12 '20 at 09:54
  • @ParthPatel: Thank you for the link. Can you please provide steps to create one after getting the source? – user2801184 Mar 13 '20 at 23:16
  • @user2801184 This is too long process not able to explain and I don't know about it. I know only one thing you have to make audio function to pass your audio stream. – Parth Patel Mar 14 '20 at 05:36
  • @ParthPatel: Please share the audio function if possible. – user2801184 Mar 15 '20 at 02:28
  • @ParthPatel@Aboulfouz: Hey Guys I am getting error with latest webrtc libraries Can you please help and answer my question? https://stackoverflow.com/questions/61190672/unable-to-screencast-to-apprtc-using-replay-kit-over-webrtc – user2801184 Apr 13 '20 at 15:03

0 Answers0