in captureOutput I am getting video frames in sample buffers and I want to alpha blend another CIImage on top of the captured frames. Rendering should directly go into the passed sample buffer. Currently I use this:
func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {
let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
let contextImage: CIImage? = CIImage(cvPixelBuffer: imageBuffer!)
let contextExtent = contextImage!.extent
// init rendering into CI context
if (m_eaglContext != EAGLContext.current()) {
EAGLContext.setCurrent(m_eaglContext)
}
m_videoPreviewView.bindDrawable()
// render overlay into pixel buffer
m_ciContext.render(m_overlayImage!, to: imageBuffer!, bounds: CGRect(x:0, y:0, width:200, height:200), colorSpace: m_colorspace)
// display the pixelbuffer into GLKView
m_ciContext.draw(contextImage!, in: m_videoPreviewViewBounds, from: contextExtent)
m_videoPreviewView.display()
}
Essentially the code works, but there are two problems:
The alpha component of m_overlayImage is ignored, i.e. the translucent parts of m_overlayImage are rendered black by the render-function. Why ? How to work around ?
Is there any way to draw m_overlayImage to a certain pixel position in the sampleBuffer. Currently I only can draw to (0,0), i.e. bottom left corner. The problem is that according to the docs 'bounds' are applied to the overlay image instead of the sample buffer. How can I render my m_overlayImage to a certain position in sample buffer ?
Thanks Chris