AVCaptureVideoDataOutput captureOutput not being called

Question

I'm trying screen capture on a Mac with AVCaptureScreenInput, but AVCaptureVideoDataOutput delegate captureOutput is never called, and I'm not sure why. I do get a notification saying the capture session was started.

import Cocoa
import AVFoundation

class ViewController: NSViewController, AVCaptureVideoDataOutputSampleBufferDelegate {

    var captureSession: AVCaptureSession!

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view.
    }

    override func viewWillAppear() {
        super.viewWillAppear()
        NSNotificationCenter.defaultCenter().addObserver(self, selector: #selector(ViewController.errorNotif), name: AVCaptureSessionRuntimeErrorNotification, object: nil)
        NSNotificationCenter.defaultCenter().addObserver(self, selector: #selector(ViewController.startedNotif), name: AVCaptureSessionDidStartRunningNotification, object: nil)
        startScreenCapture()
    }

    override func viewWillDisappear() {
        super.viewWillDisappear()
        NSNotificationCenter.defaultCenter().removeObserver(self)
    }

    func captureOutput(captureOutput: AVCaptureOutput!, didDropSampleBuffer sampleBuffer: CMSampleBuffer!, fromConnection connection: AVCaptureConnection!) {
        print("ignore frame, add code to handle later")
    }

    func startScreenCapture() {

        let displayId = CGMainDisplayID()
        captureSession = AVCaptureSession()
        if captureSession.canSetSessionPreset(AVCaptureSessionPresetHigh) {
            captureSession.sessionPreset = AVCaptureSessionPresetHigh
        }
        let captureScreenInput = AVCaptureScreenInput(displayID: displayId)
        if captureSession.canAddInput(captureScreenInput) {
            captureSession.addInput(captureScreenInput)
        } else {
            print("Could not add main display to capture input")
        }

        let output = AVCaptureVideoDataOutput()

        let queue = dispatch_queue_create("myQueue", DISPATCH_QUEUE_SERIAL)
        output.setSampleBufferDelegate(self, queue: queue)
        output.alwaysDiscardsLateVideoFrames = true

        output.videoSettings = [kCVPixelBufferPixelFormatTypeKey as NSString: NSNumber(unsignedInt: kCVPixelFormatType_32BGRA)]
        captureSession.addOutput(output)
        captureSession.startRunning()
    }

    func errorNotif() {
        print("error starting capture")
    }
    func startedNotif() {
        print("started screen capture")
    }
}

score 3 · Answer 1 · answered Nov 30 '19 at 12:49

I added for basic example AVCaptureVideoDataOutputSampleBufferDelegate

class ViewController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate {    


    let videoQueue = DispatchQueue(label: "VIDEO_QUEUE")

    override func viewDidLoad() {
        super.viewDidLoad()

        let captureSession = AVCaptureSession()

        guard let captureDevice = AVCaptureDevice.default(for: .video) else { return }

        guard let input = try? AVCaptureDeviceInput(device: captureDevice) else { return }

        captureSession.addInput(input)

        captureSession.startRunning()

        let previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
        view.layer.addSublayer(previewLayer)
        previewLayer.frame = view.frame

        let dataOutput = AVCaptureVideoDataOutput()
        dataOutput.setSampleBufferDelegate(self, queue: videoQueue)
        captureSession.addOutput(dataOutput)
    }

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        print("Camera was able to capture a frame:", Date())
    }
}

Rhythmic Fistman · Accepted Answer · 2016-06-17T23:31:02.163

2

You need to define the didOutputSampleBuffer delegate callback to actually receive the captured frames:

func captureOutput(captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, fromConnection connection: AVCaptureConnection!) {
    print("captured \(sampleBuffer)")
}

p.s. I'm not sure about macOS, but viewWillAppear may not be a good place to do initialisation because on iOS at least it can be called multiple times.

edited Jun 17 '16 at 23:31

answered Jun 17 '16 at 23:27

Rhythmic Fistman

34,352
5
87
159

Thank you so much! I chose the wrong captureOutput from autocomplete. Agree, viewWillAppear is not a good place to call start. My real app will call it based on user interaction – Ravi Jun 17 '16 at 23:38
I have a similar problem, but I'm not using a ViewController in my app as I'm using GTK instead. I have `public class Vision : NSObject, AVCaptureVideoDataOutputSampleBufferDelegate` with `captureOutput` defined in it. In its constructor, I am setting the `setSampleBufferDelegate` to `(self , queue: queue)`. My code compiles, runs, and displays the GUI. However, whilst the built-in camera does turn on, it stays on momentarily and then turns off, but `captureOutput` is never called. – gone Feb 18 '18 at 15:40
Can you start a new question and show how you set up the capture session? – Rhythmic Fistman Feb 18 '18 at 17:34
@gone Did you find a solution to your problem? I have the same issue – Manish Kumar Dec 17 '18 at 07:53
@ManishKumar: I abandoned my efforts at the time. Due to your comment, I came back to it, and, after getting the privacy permissions working, have found that `captureOutput` is being called, though I get an error after a short time. I've put my code in pastebin: https://pastebin.com/0Ew9as5x, but if you can't reach and there is some way you can contact me directly, I can share it with you. – gone Dec 21 '18 at 07:27
@ManishKumar: The `DispatchQueue.main.async` inside the callback isn't being called, but I don't have time right now to debug that part of it. – gone Dec 21 '18 at 07:34
@gone Thank you for your effort. I was able to fix it. Turned out to be good old `scope` issue. – Manish Kumar Dec 22 '18 at 10:38

AVCaptureVideoDataOutput captureOutput not being called

2 Answers2

Linked