When creating a lot of thumbnails from video file the AVAssetImageGenerator throws error

Question

I have problem with AVAssetImageGenerator. Quality of thumbnail is middle, size of pictures on the average around 50 kilobytes. I create a lot of thumbnails from video file my code:

- (UIImage *)generateThumbnailImage: (NSString *)srcVideoPath atTime:(CMTime)time
    {
        NSURL *url = [NSURL fileURLWithPath:srcVideoPath];

        if ([srcVideoPath rangeOfString:@"://"].location == NSNotFound)
        {
            url = [NSURL URLWithString:[[@"file://localhost" stringByAppendingString:srcVideoPath] stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]];
        }
        else
        {
            url = [NSURL URLWithString:[srcVideoPath stringByAddingPercentEscapesUsingEncoding: NSUTF8StringEncoding]];
        }

        AVAsset *asset = [AVAsset assetWithURL:url];
        AVAssetImageGenerator *imageGenerator = [[AVAssetImageGenerator alloc] initWithAsset:asset];
        imageGenerator.requestedTimeToleranceAfter = kCMTimeZero; // needed to get a precise time (http://stackoverflow.com/questions/5825990/i-cannot-get-a-precise-cmtime-for-generating-still-image-from-1-8-second-video)
        imageGenerator.requestedTimeToleranceBefore = kCMTimeZero; // ^^
        imageGenerator.appliesPreferredTrackTransform = YES; // crucial to have the right orientation for the image (http://stackoverflow.com/questions/9145968/getting-video-snapshot-for-thumbnail)
        NSError *error;
        CMTimeAdd(kCMTimeZero, asset.duration);
        CGImageRef imageRef = [imageGenerator copyCGImageAtTime:time actualTime:NULL error:&error];

        UIImage *thumbnail = [UIImage imageWithCGImage:imageRef];
        CGImageRelease(imageRef);  // CGImageRef won't be released by ARC
        NSLog(@"error: %@", error);
        return thumbnail;
    } 

- (void) createThumbnail: (NSString *)srcVideoPath atTime:(CMTime)time (NSInteger *) width:(NSInteger *) height:(NSInteger *)thumbQuality
{                                                             
    //Image processing                                                             
    UIImage* thumbnail = [self generateThumbnailImage:srcVideoPath atTime:time];


    CGSize newSize = CGSizeMake(width, height);
    thumbnail = [self scaleImage:thumbnail toSize:newSize];

    NSString *cacheDir = [NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES) objectAtIndex:0];
    NSString *outputFilePath = [cacheDir stringByAppendingPathComponent:[NSString stringWithFormat:@"%@.%@", outputFileName, @"jpg"]];

    [UIImageJPEGRepresentation(thumbnail, thumbQuality) writeToFile:outputFilePath atomically:YES]                               
}

- (UIImage*)scaleImage:(UIImage*)image toSize:(CGSize)newSize;{
    float oldWidth = image.size.width;
    float scaleFactor = newSize.width / oldWidth;

    float newHeight = image.size.height * scaleFactor;
    float newWidth = oldWidth * scaleFactor;

    UIGraphicsBeginImageContext(CGSizeMake(newWidth, newHeight));
    [image drawInRect:CGRectMake(0, 0, newWidth, newHeight)];
    UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    return newImage;
}

The CGImageRef log gives error after creating 294 thumbnails:

Error Domain=AVFoundationErrorDomain Code=-11839 "Cannot Decode" UserInfo={NSLocalizedRecoverySuggestion=Stop any other actions that decode media and try again., NSLocalizedFailureReason=The decoder required for this media is busy., NSLocalizedDescription=Cannot Decode, NSUnderlyingError=0x1000d660 {Error Domain=NSOSStatusErrorDomain Code=-12913 "(null)"}}

How to fix it? How to release or stop any actions of decoder?

I'm using this code in cordova plugin. I have loop by javascript to generate thumbnails for specific time. I have tried with generateCGImagesAsynchronouslyForTimes I got same result.

Answer 1

The following worked for me. No additional resizing code is required.

static func fetchThumbnail(video: AVAsset, maxSize: CGSize = CGSize.zero, time: CMTime = .zero) -> UIImage? {
    let imgGenerator = AVAssetImageGenerator(asset: video)
    imgGenerator.appliesPreferredTrackTransform = true
    imgGenerator.maximumSize = CGSize.init(width: maxSize.width * UIScreen.main.scale, height: maxSize.height * UIScreen.main.scale)
    do {
        let cgImage: CGImage = try imgGenerator.copyCGImage(at: CMTimeAdd(CMTime.zero, time), actualTime: nil)
        return UIImage.init(cgImage: cgImage)
    } catch {
        return nil
    }
}

Pass an array of NSValue objected created using CMTime objects to generate a series of thumbnails

static func fetchThumbnails(video: AVAsset, maxSize: CGSize = CGSize.zero, times: [NSValue], completion: @escaping ((_ time: CMTime, _ image: CGImage?, _ actualTime: CMTime, _ result: AVAssetImageGenerator.Result, _ error: Error?) -> Void)) {
    let imgGenerator = AVAssetImageGenerator(asset: video)
    imgGenerator.appliesPreferredTrackTransform = true
    imgGenerator.maximumSize = CGSize.init(width: maxSize.width * UIScreen.main.scale, height: maxSize.height * UIScreen.main.scale)
    imgGenerator.generateCGImagesAsynchronously(forTimes: times) { (time: CMTime, image: CGImage?, actualTime: CMTime, result: AVAssetImageGenerator.Result, error: Error?) in
        DispatchQueue.main.async {
            completion(time, image, actualTime, result, error)
        }
    }
}