Unable to draw CIImage on GLKView after few frames since updated to iOS 10.2?

Question

Using following code in my application which was performing quiet fine to draw a CIImage on a GLKView again and again as recieved from AVCaptureOutput -didOutputSampleBuffer until I was using iOS <= 10.1.*

After updating device to iOS 10.2.1 it has stopped working. I am calling it for few frames the app just crashes with low memory warning. Whereas with iOS 10.1.1 and below I smoothly runs the app even on older device like iPhone 5S.

[_glkView bindDrawable];  

if (self.eaglContext != [EAGLContext currentContext])  
[EAGLContext setCurrentContext:self.eaglContext];  

glClearColor(0.0, 0.0, 0.0, 1.0);  
glClear(GL_COLOR_BUFFER_BIT);  

glEnable(GL_BLEND);  
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);  

if (ciImage) {  
    [_ciContext drawImage:ciImage inRect:gvRect fromRect:dRect];  
}  

[_glkView display];

This is how I am making the CIImage.

- (CIImage*)ciImageFromPixelBuffer:(CVPixelBufferRef)pixelBuffer ofSampleBuffer:(CMSampleBufferRef)sampleBuffer {
CIImage *croppedImage           = nil;

CFDictionaryRef attachments     = CMCopyDictionaryOfAttachments(kCFAllocatorDefault, sampleBuffer, kCMAttachmentMode_ShouldPropagate);
CIImage *ciImage                = [CIImage imageWithCVPixelBuffer:pixelBuffer options:(NSDictionary *)attachments];

if (attachments)
    CFRelease(attachments);

croppedImage = ciImage;


    CIFilter *scaleFilter = [CIFilter filterWithName:@"CILanczosScaleTransform"];
    [scaleFilter setValue:croppedImage forKey:@"inputImage"];
    [scaleFilter setValue:[NSNumber numberWithFloat:self.zoom_Resize_Factor == 1 ? 0.25 : 0.5] forKey:@"inputScale"];
    [scaleFilter setValue:[NSNumber numberWithFloat:1.0] forKey:@"inputAspectRatio"];
    croppedImage = [scaleFilter valueForKey:@"outputImage"];


    NSDictionary *options = @{(id)kCIImageAutoAdjustRedEye : @(false)};

    NSArray *adjustments = [ciImage autoAdjustmentFiltersWithOptions:options];
    for (CIFilter *filter in adjustments) {
        [filter setValue:croppedImage forKey:kCIInputImageKey];
        croppedImage = filter.outputImage;
    }

CIFilter *selectedFilter = [VideoFilterFactory getFilterWithType:self.selectedFilterType]; //This line needs to be removed from here

croppedImage = [VideoFilterFactory applyFilter:selectedFilter OnImage:croppedImage];

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

return croppedImage;
}

Here is imgur link https://i.stack.imgur.com/wQ4mP.jpg of VM Tracker and OpenGL ES instruments result. Incase it eases to understand. Thanks.

What happens if you bypass the CIFilter related processing ? — Sabir Ali, Feb 10 '17 at 11:03
Okay I passed pixelbuffer as it is to glkview and it did not crash or has any effect. But why it happened so? — khunshan, Feb 10 '17 at 11:07

score 1 · Accepted Answer · answered Feb 10 '17 at 11:26

1

Your GLKView rendering implementation looks fine, the issue seems to be coming from the amount of processing you're doing on PixelBuffer after converting it into CIImage.

Also the Imgur link you shared shows that GLKView is unable to prepare VideoTexture object correctly, most probably due to the memory overload being created in each iteration. You need to optimise this CIFilter Processing.

answered Feb 10 '17 at 11:26

Sabir Ali

475
2
16

Does it mean i can never use some of filters. I looked at CIFunhouse they are implementing almost all of Core Image filters but..... confused. – khunshan Feb 10 '17 at 11:36
Check with the resolution of your PixelBuffer if its same as in the CIFunhouse, more over every CIFilters cannot be applied at 24 or 30 frames per second, and so aren't suitable for videos at higher resolutions, specially on last year or second last year iOS devices. – Sabir Ali Feb 10 '17 at 12:04
What way I can put all filters in working condition. Should I scale down or use low preset? – khunshan Feb 10 '17 at 12:36

Unable to draw CIImage on GLKView after few frames since updated to iOS 10.2?

1 Answers1