0

I'm trying to stream a CMSampleBuffer video / audio combo using WebRTC on iOS, but I'm running into trouble trying to capture audio. Video works just fine:

guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
    print("couldn't get image from buffer :~(")
    
    return
}
        
let rtcPixelBuffer = RTCCVPixelBuffer(pixelBuffer: pixelBuffer)
let rtcVideoFrame = RTCVideoFrame(buffer: rtcPixelBuffer, rotation: ._0, timeStampNs: timeStampNs)

videoSource.capturer(videoCapturer, didCapture: rtcVideoFrame)

When it comes to audio, I can't see any method on the RTCAudioSource class in order to capture audio, any help would be appreciated!

William
  • 81
  • 1
  • 9

1 Answers1

1

I found a fork of the WebRTC codebase which solves this issue by adding a way for audio samples to be captured by an RTCAudioDeviceModule:

https://github.com/pixiv/webrtc/blob/87.0.4280.142-pixiv0/README.pixiv.en.md

William
  • 81
  • 1
  • 9
  • Hi william, I currently have the same problem and being strugling for days to solve it. I added WebRTC.framework from this fork to my project. Video is working ok, but I cant get results with audio. Can you please share your implementation for sending audio from CMSampleBuffer? I see there is some adjustment with RTCAudioSession.useManualAudio setting to YES and RTCAudioSession.isAudioEnabled setting to NO in Broadcast Upload Extension. It would be great if you could write steps how you achieved sending audio from CMSampleBuffer using the code from this WebRTC fork. Thanks in advance! – Mile Dev Jan 28 '22 at 14:45
  • @MileDev Hey Mile, with this fork you should be able to use `RTCAudioDeviceModule.deliverRecordedData` and pass in the sample buffer there. It's been a while since I worked on this project so I'm not sure what else you need to do apart from setting `RTCAudioSession.useManualAudio` to `YES and `RTCAudioSession.isAudioEnabled` to `NO`. – William Jan 29 '22 at 19:47
  • Thank you for your answer. I already implemented their fork and added deliverRecordedData method per their guideline. Also set .useManualAudio and .isAudioEnabled per requirements. From some reason, when app is backgrounded, outgoing audio stops working. Let me know if you have any advice regarding that. Thanks again for your response! – Mile Dev Jan 31 '22 at 09:22