AVFoundation - how to mirror video from webcam - Mac OS X

Question

I am trying to mirror video received from webcam on mac os x. I would like to avoid doing a manual flip/tranform after receiving the video buffer . So, I want to setup AVCaptureSession such that video buffer received in captureOutput method of AVCaptureVideoDataOutputSampleBufferDelegate is mirrored by AVFoundation itself. I don't want to use the preview layer.

On an iMac(10.8.5), to mirror video, AVCaptureConnection isVideoMirroringSupported is successfully tested before setting the videoMirrored property. But video buffer received in captureOutput delegate isn't mirrored.

Note: Video mirroring on iOS was successful, when I followed this SO answer. But it isn't helping on mac os x.

Code used is below. Error checking is left out for this post.

    //create session
    _session = [[AVCaptureSession alloc] init];

    //get capture device
    _device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];

    //create sesion input
    NSError * error;
    _sessionInput = [AVCaptureDeviceInput deviceInputWithDevice:_device error:&error];

    //create session output
    _sessionOutput = [[AVCaptureVideoDataOutput alloc] init];
    [_sessionOutput setAlwaysDiscardsLateVideoFrames:YES];
    [[_sessionOutput connectionWithMediaType:AVMediaTypeVideo] setEnabled:YES];
    NSDictionary *videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey];
    [_sessionOutput setVideoSettings:videoSettings];

    //serial queue to process video frames
    dispatch_queue_t videoOutputQueue = dispatch_queue_create("deviceeraQueue", DISPATCH_QUEUE_SERIAL);
    [_sessionOutput setSampleBufferDelegate:self queue:videoOutputQueue];

    //begin session configuration
    [_session beginConfiguration ];

    //input and output for session
    if( [_session canAddInput:_sessionInput]) {
        [_session addInput:_sessionInput];
    }
    if( [_session canAddOutput:_sessionOutput]) {
        [_session addOutput:_sessionOutput];

    }

    //set video mirroring
    AVCaptureConnection* avConnection = [_sessionOutput connectionWithMediaType:AVMediaTypeVideo];
    if( [avConnection isVideoMirroringSupported]) {
        avConnection.videoMirrored = YES;
        NSLog(@"Video mirroring Support: YES"); // this line is printed
    } else {
        NSLog(@"Video mirroring Support: NO");
    }

    //set session preset    
    [_session setSessionPreset:AVCaptureSessionPreset640x480];
    [ _session commitConfiguration ];

    ...........
    ...........

    - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
    {
    .........
    //sampleBuffer is not mirrored video
    ........

Of lesser importance 1 - though C++, I also tried looking into OpenCV's VideoCapture implementation for way to mirror video. But, OpenCV don't mirror video from Mac(uses flip). Left is libVlc/V4L.

Of lesser importance 2 - In slide 73 of this 2010 wwdc apple presentation (3Mb pdf), there is a mention that setVideoOrientation is not supported on 'AVCaptureVideoDataOutput` connection. But in 2013, apple docs are updated and supports this method.

Found the same behavior on the `isVideoMirrored` API, I uses a preview layer, the preview layer can be set mirrored correctly but I used an `AVCaptureAudioDataOutput` to actually record the video, in the output delegate: `captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection)` the sample buffer is not mirrored. — kakaiikaka, Oct 12 '21 at 06:03

score 6 · Answer 1 · answered Oct 23 '14 at 16:47

6

You can add a transform on the preview layer to flip x value of the frames before they get to the preview window.

[[self previewLayer] setTransform:CATransform3DMakeScale(-1, 1, 1)];

Then you can run the recorded video through export session and do the same transformation. That way the video preview will match the final recorded video. Bit of a hack, but it gets the same results.

answered Oct 23 '14 at 16:47

tansk1

186
3
8

Thanks. But in question, I had mentioned about reluctance to use preview layer ( in paragraph 1 ) – kiranpradeep Oct 24 '14 at 05:05
2

OSX does not respect the videoMirrored property, probably a bug. Using a preview layer is the only way I've found to achieve the effect. – tansk1 Oct 24 '14 at 16:23

score 5 · Answer 2 · answered Jun 07 '15 at 17:15

5

Why hack it when it's very easy. Just set automaticallyAdjustVideoMirroring of your AVCaptureConnection then set it manually.

    aPreviewLayer.connection.automaticallyAdjustsVideoMirroring = NO;
    aPreviewLayer.connection.videoMirrored = YES;

answered Jun 07 '15 at 17:15

Þorvaldur Rúnarsson

879
7
15

Thanks. But kindly see my mention about reluctance to you preview layer ( in paragraph 1 of question). The same is again mentioned in my comment to the earlier answer. – kiranpradeep Jun 08 '15 at 05:08
I don't think you can without using the preview layer. But you can certainly achieve the same effect by for example scaling. Scaling buffer by (x,y,z) = (-1,1,1) would achieve the mirroring affect :) – Þorvaldur Rúnarsson Jun 08 '15 at 13:19
`I don't think you can without using the preview layer.` - Then to close the question, to support that, we will need a valid source document or an accepted bug report by Apple. – kiranpradeep Jun 08 '15 at 13:37

score 0 · Answer 3 · answered Jun 11 '20 at 08:42

0

Swift 5 version of "Þorvaldur Rúnarsson" answer:

previewLayer.connection?.automaticallyAdjustsVideoMirroring = false
previewLayer.connection?.isVideoMirrored = true

answered Jun 11 '20 at 08:42

Ahmadreza

6,950
5
50
69

AVFoundation - how to mirror video from webcam - Mac OS X

3 Answers3