Generating custom thumbnail from ALAssetRepresentation

Question

My main problem is i need to obtain a thumbnail for an ALAsset object.

I tried a lot of solutions and searched stack overflow for days, all the solutions i found are not working for me due to these constraint:

• I can't use the default thumbnail because it's too little;
• I can't use the fullScreen or fullResolution image because i have a lot of images on screen;
• I can't use UIImage or UIImageView for resizing because those loads the fullResolution image
• I can't load the image in memory, i'm working with 20Mpx images;
• I need to create a 200x200 px version of the original asset to load on screen;

this is the last iteration of the code i came with:

#import <AssetsLibrary/ALAsset.h>
#import <ImageIO/ImageIO.h>   

// ...

ALAsset *asset;

// ...

ALAssetRepresentation *assetRepresentation = [asset defaultRepresentation];

NSDictionary *thumbnailOptions = [NSDictionary dictionaryWithObjectsAndKeys:
    (id)kCFBooleanTrue, kCGImageSourceCreateThumbnailWithTransform,
    (id)kCFBooleanTrue, kCGImageSourceCreateThumbnailFromImageAlways,
    (id)[NSNumber numberWithFloat:200], kCGImageSourceThumbnailMaxPixelSize,
    nil];

CGImageRef generatedThumbnail = [assetRepresentation CGImageWithOptions:thumbnailOptions];

UIImage *thumbnailImage = [UIImage imageWithCGImage:generatedThumbnail];

problem is, the resulting CGImageRef is neither transformed by orientation, nor of the specified max pixel size;

I also tried to find a way of resizing using CGImageSource, but:

• the asset url can't be used in the CGImageSourceCreateWithURL:;
• i can't extract from ALAsset or ALAssetRepresentation a CGDataProviderRef to use with CGImageSourceCreateWithDataProvider:;
• CGImageSourceCreateWithData: requires me to store the fullResolution or fullscreen asset in memory in order to work.

Am i missing something?

Is there another way of obtaining a custom thumbnail from ALAsset or ALAssetRepresentation that i'm missing?

Thanks in advance.

Answer 1

You can use CGImageSourceCreateThumbnailAtIndex to create a small image from a potentially-large image source. You can load your image from disk using the ALAssetRepresentation's getBytes:fromOffset:length:error: method, and use that to create a CGImageSourceRef.

Then you just need to pass the kCGImageSourceThumbnailMaxPixelSize and kCGImageSourceCreateThumbnailFromImageAlways options to CGImageSourceCreateThumbnailAtIndex with the image source you've created, and it will create a smaller version for you without loading the huge version into memory.

I've written a blog post and gist with this technique fleshed out in full.

Answer 2

There is a problem with this approach mentioned by Jesse Rusak. Your app will be crashed with the following stack if asset is too large:

0   CoreGraphics              0x2f602f1c x_malloc + 16
1   libsystem_malloc.dylib    0x39fadd63 malloc + 52
2   CoreGraphics              0x2f62413f CGDataProviderCopyData + 178
3   ImageIO                   0x302e27b7 CGImageReadCreateWithProvider + 156
4   ImageIO                   0x302e2699 CGImageSourceCreateWithDataProvider + 180
...

Link Register Analysis:

Symbol: malloc + 52

Description: We have determined that the link register (lr) is very likely to contain the return address of frame #0's calling function, and have inserted it into the crashing thread's backtrace as frame #1 to aid in analysis. This determination was made by applying a heuristic to determine whether the crashing function was likely to have created a new stack frame at the time of the crash.

Type: 1

It is very easy to simulate the crash. Let's read data from ALAssetRepresentation in getAssetBytesCallback with a small chunks. The particular size of the chunk is not important. The only thing which matters is calling callback about 20 times.

static size_t getAssetBytesCallback(void *info, void *buffer, off_t position, size_t count) {
    static int i = 0; ++i;
    ALAssetRepresentation *rep = (__bridge id)info;
    NSError *error = nil;
    NSLog(@"%d: off:%lld len:%zu", i, position, count);
    const size_t countRead = [rep getBytes:(uint8_t *)buffer fromOffset:position length:128 error:&error];
    return countRead;
}

Here are tail lines of the log

2014-03-21 11:21:14.250 MRCloudApp[3461:1303] 20: off:2432 len:2156064

MRCloudApp(3461,0x701000) malloc: *** mach_vm_map(size=217636864) failed (error code=3)

*** error: can't allocate region

*** set a breakpoint in malloc_error_break to debug

I introduced a counter to prevent this crash. You can see my fix below:

typedef struct {
    void *assetRepresentation;
    int decodingIterationCount;
} ThumbnailDecodingContext;
static const int kThumbnailDecodingContextMaxIterationCount = 16;

static size_t getAssetBytesCallback(void *info, void *buffer, off_t position, size_t count) {
    ThumbnailDecodingContext *decodingContext = (ThumbnailDecodingContext *)info;
    ALAssetRepresentation *assetRepresentation = (__bridge ALAssetRepresentation *)decodingContext->assetRepresentation;
    if (decodingContext->decodingIterationCount == kThumbnailDecodingContextMaxIterationCount) {
        NSLog(@"WARNING: Image %@ is too large for thumbnail extraction.", [assetRepresentation url]);
        return 0;
    }
    ++decodingContext->decodingIterationCount;
    NSError *error = nil;
    size_t countRead = [assetRepresentation getBytes:(uint8_t *)buffer fromOffset:position length:count error:&error];
    if (countRead == 0 || error != nil) {
        NSLog(@"ERROR: Failed to decode image %@: %@", [assetRepresentation url], error);
        return 0;
    }
    return countRead;
}

- (UIImage *)thumbnailForAsset:(ALAsset *)asset maxPixelSize:(CGFloat)size {
    NSParameterAssert(asset);
    NSParameterAssert(size > 0);
    ALAssetRepresentation *representation = [asset defaultRepresentation];
    if (!representation) {
        return nil;
    }
    CGDataProviderDirectCallbacks callbacks = {
        .version = 0,
        .getBytePointer = NULL,
        .releaseBytePointer = NULL,
        .getBytesAtPosition = getAssetBytesCallback,
        .releaseInfo = NULL
    };
    ThumbnailDecodingContext decodingContext = {
        .assetRepresentation = (__bridge void *)representation,
        .decodingIterationCount = 0
    };
    CGDataProviderRef provider = CGDataProviderCreateDirect((void *)&decodingContext, [representation size], &callbacks);
    NSParameterAssert(provider);
    if (!provider) {
        return nil;
    }
    CGImageSourceRef source = CGImageSourceCreateWithDataProvider(provider, NULL);
    NSParameterAssert(source);
    if (!source) {
        CGDataProviderRelease(provider);
        return nil;
    }
    CGImageRef imageRef = CGImageSourceCreateThumbnailAtIndex(source, 0, (__bridge CFDictionaryRef) @{(NSString *)kCGImageSourceCreateThumbnailFromImageAlways : @YES,
                                                                                                      (NSString *)kCGImageSourceThumbnailMaxPixelSize          : [NSNumber numberWithFloat:size],
                                                                                                      (NSString *)kCGImageSourceCreateThumbnailWithTransform   : @YES});
    UIImage *image = nil;
    if (imageRef) {
        image = [UIImage imageWithCGImage:imageRef];
        CGImageRelease(imageRef);
    }
    CFRelease(source);
    CGDataProviderRelease(provider);
    return image;
}