CGImage from images stored on iCloud look wrong

Question

I'm extracting pixel colors from a CGImage using the code described in this answer.

However, I just realized that if I load an image that was created on another device, the pixels values look wrong. The first obvious problem is that the alpha is gone. The CGImageAlphaInfo reports .noneSkipLast, but I know the image is RGBA. If I read it from the same device it was created, it looks fine. The second problem is that there is some color bleeding, as if the image had been resized. Perhaps is being compressed or something.

Here's an example:

Source image is this watermelon, 12x12

It was created on my iPad. But if I load it on my iPhone through iCloud using that code I linked, I get this:

The alpha channel is gone, and colors bleed.

If from the same iPhone I send the little watermelon to my Mac using Airdrop, and send it back using Airdrop again (so it is supposedly the same image!), and load it now, I get the correct image:

(dark brown areas is where alpha is 0)

If you have a couple of iOS device with iCloud enabled, you can reproduce this behavior in this app: http://swiftpixels.endavid.com where I'm using that code to read pixel colors.

What could be the difference between those images? How can I read the correct image from iCloud? Should I look for hints in UIImage instead of CGImage?

Any clues? Thanks!

Update

For reference, I'm reading the image using a UIImagePickerController, using this code:

    func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
        if let image = info[UIImagePickerController.InfoKey.originalImage] as? UIImage {
            loadImageInMemory(image)
        }
        picker.dismiss(animated: true, completion: nil)
        self.presentingViewController?.dismiss(animated: true, completion: nil)
    }

    fileprivate func loadImageInMemory(_ image: UIImage) {
       /// skipping some preparation
       guard let cgImage = image.cgImage else {
          return
       }
       /// getImageData is a function like getPixelColor linked above,
       /// but for a Rect 
       self.imageData = cgImage.getImageData(rect, width, height)
    }

I also found this question which may be related: UIImagePickerController and iCloud photos

As Rob suggested in the comments below, I changed the Photos settings in my phone to "Download and Keep Originals" (instead of "Optimize iPhone Storage"), and that fixes the problem. So I guess the question is why iCloud tries to compress a PNG image that it's just 1,355 bytes, and whether it's possible to access the original image from the UIImagePickerController

“... if I load an image that was created on another device” ... How precisely are you doing that? What sort of representation are you exchanging between these devices? PNGs? JPGs? Bottom line, the problem isn’t `UIImage` vs `CGImage`, but what sort file representation/payload you’re exchanging. — Rob, Feb 17 '20 at 05:35
It's the PNG I posted there. JPGs do not have transparency, unless you use Jpeg2000. The PNG was created using the app I linked. It is saved on iPad to the Photos folder, and then it appears on the iPhone because I have iCloud Photos enabled. It says: "Automatically upload and safely store all your photos and videos in iCloud so you can browse, search, and share from any of your devices". It doesn't look like iCloud converts the PNG into a Jpeg because if I Airdrop it from the iPhone to my Mac, I can see it's still a PNG (the little watermelon attached there on my question). — endavid, Feb 17 '20 at 08:41
Yes! Thank you :) If I set it to "Download and keep originals", then the image looks correct. That `12x12` watermelon is a PNG of just 1,355 bytes, so I don't understand what compression gets applied to it... It seems Airdrop can access the original image, though. How can I make my app access the original as well? — endavid, Feb 17 '20 at 22:56
I found a related question, I think, https://stackoverflow.com/q/25012731/1765629 -- I've updated my question with that link and some code. I used the Photos Library for convenience, and I thought it would be a better experience for the users of the app. The app has been around since 2016, and I just noticed this problem now, so I guess it's not a major inconvenience. It would be nice to understand what's going on, though. — endavid, Feb 17 '20 at 23:13

score 1 · Answer 1 · answered Feb 17 '20 at 23:15

In the Settings app, under “Photos” there is an “Optimize iPhone Storage” vs “Download and Keep Originals” option. The latter should synchronize the photos library without altering the images in any way.

It seems Airdrop can access the original image, though.

Airdrop is not relying upon photo library iCloud synchronization to send the image. That's why the asset makes it across, unaltered.

How can I make my app access the original as well?

It comes down to whether you really want to use the Photos library to synchronize your assets. This seems especially true as many people may not choose to sync their huge Photos library to iCloud at all. I'm not sure you want to rely upon that.

You might want to just use CloudKit yourself rather than relying upon the Photos library. Perhaps you should have an option in the app so a user can choose to use iCloud storage at all, and if so, do the iCloud storage within your app rather than relying upon the Photos library.

endavid · Accepted Answer · 2020-10-29T23:27:25.483

The reason why the image looked blurry and without alpha seems to be that Photos streams Jpeg-compressed images from iCloud by default. Even if your original image would be smaller without compression, like in this pixel art example.

As pointed out by Rob, a way to verify that this is the case is to change your Photos settings. From the Settings app:

Settings -> Photos -> Download and Keep Originals

This would fix the issue, but of course it's not desirable. If you want to keep using Photos, instead of implementing your own iCloud solution, while keeping the Optimize iPhone Storage setting, you can use PhotoKit to retrieve the original image.

Replace this code,

    func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
        if let image = info[UIImagePickerController.InfoKey.originalImage] as? UIImage {
            loadImageInMemory(image)
        }
        picker.dismiss(animated: true, completion: nil)
        self.presentingViewController?.dismiss(animated: true, completion: nil)
    }

by this other code:

import Photos

// ...

    func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
        // it will be loaded asynchronously
        loadImageFromPicker(info: info)
        picker.dismiss(animated: true, completion: nil)
        self.presentingViewController?.dismiss(animated: true, completion: nil)
    }

    private func loadImageFromPicker(info: [UIImagePickerController.InfoKey : Any]) {
        var phAsset: PHAsset?
        if #available(iOS 11.0, *) {
            phAsset = info[UIImagePickerController.InfoKey.phAsset] as? PHAsset
        } else {
            // Fallback on earlier versions
            if let referenceURL = info[UIImagePickerController.InfoKey.referenceURL] as? URL {
                let fetchResult = PHAsset.fetchAssets(withALAssetURLs: [referenceURL], options: nil)
                phAsset = fetchResult.firstObject
            }
        }
        guard let asset = phAsset else {
            return
        }
        // size doesn't matter, because resizeMode = .none
        let size = CGSize(width: 32, height: 32)
        let options = PHImageRequestOptions()
        options.version = .original
        options.deliveryMode = .highQualityFormat
        options.resizeMode = .none
        options.isNetworkAccessAllowed = true
        PHImageManager.default().requestImage(for: asset, targetSize: size, contentMode: .aspectFit, options: options) { [weak self] (image, info) in
            if let s = self, let image = image {
                s.loadImageInMemory(image)
            }
        }
    }

This code will work with both local images and iCloud images.

CGImage from images stored on iCloud look wrong

2 Answers2

Linked