Why in Swift5, the UIImage.jpegData() call will retain that much memory?

Question

I had an application that runs the following code snippets --

...
let newImage = generateImage()
let imageData = newImage.jpegData(compressionQuality: 1)

// operations with the imageData. The memory consumption of the
// imageData object looks reasonable, 2-3 MB for a 4800*6000
// pixel image.
...

func generateImage() -> UIImage {
    let canvasSize = self.outputImage.size
    UIGraphicsBeginImageContext(canvasSize)
    self.imageView.zoomView?.layer.render(in: UIGraphicsGetCurrentContext()!)
    let newImage = UIGraphicsGetImageFromCurrentImageContext()
    UIGraphicsEndImageContext()
    return newImage!
}

The observation is that at the step of the following codes, I will see a memory spike

let imageData = outputImage.jpegData(compressionQuality: 1)

And this memory spike will not disappear until the whole view is released. I do have tried some autoreleasepool() or other put in DispatchQueue() operation as suggested in here swift UIGraphicsGetImageFromCurrentImageContext can't release memory but apparently they do not work.

To reduce confusion, the generateImage() call does not produce memory leak but just spike, i.e., the memory goes down right after the generateImage() call, but I am just putting it there to see if this related (I tried some other UIImage with the jpegData() but I cannot reproduce this issue 100% though).

I hesitate to call this a memory leak as I believe this object (I actually do not know why the UIImage.jpegData() call consumed that much memory too, looks like a bug with Swift memory management algorithm) is finally recycled when the view is unloaded. But is there any way to avoid the huge memory consumption or release the memory caused by the jpegData conversion?

(The memory will go up ~100MB with this UIImage.jpegData() operation when loading 24MB image on my iPhone with 4800*6000Pixels).

Can you please clarify how your code snippet is related to your "view". Is the code above in a subclass of UIView? — idmean, Sep 06 '20 at 14:35
Don't show "snippets". Show us the actual code, in context. We have no idea how your code runs or where, where the image comes from or where the data goes. — matt, Sep 06 '20 at 14:38

Rob Napier · Answer 1 · 2020-09-06T15:01:04.107

4800*6000 = 28.8M, @ 4 bytes per pixel (RGBA) = 115 million bytes. So ~100MB sounds exactly correct for the uncompressed data, which is exactly what you created in generateImage. CoreGraphics tries to be lazy about these things, so you may not see the actual cost until you force it to render everything (when you ask for jpegData).

Your code, as written, doesn't really make sense (you never define outputImage or use newImage), so we have to guess a lot. It's possible that you're storing the newImage or outputImage somewhere and not showing us that. Perhaps in a property, which would match "when this view is released."

But you also appear to be displaying this image with imageView. If so, that's going to eat about 100MB since it has to uncompress it to draw it on the screen. So it's very possible you're just looking at the memory usage of your image view, which would also match "when this view is released." If that's the case, I would expect the memory usage to spike to more than twice that (since you also render it, and then compress it). But you may not see that spike, since the memory would be released when you're done with it, leaving just the 100MB for the image view.

(This "100MB for the image view" is very dependent on exactly how you've set up your image view. If it's reading directly from a file, it is likely to have memory mapped the compressed data, and should only render to the actual size of the view. UIImageView is very smart. But displaying a large image is always going to cost a lot of memory. Eventually you need the bytes.)

It's a bit unclear what this code is trying to do. I suspect it could be done a little more efficiently (given that you already seem to have an image view, I'm not certain why you're rendering; you should be able to just manipulate the underlying image directly).

But ultimately, if you want a 4800*6000 pixel image in RGBA, that's going to require about 100MB of working memory to manipulate. That's the cost of large images. If it's too much memory, you'll need to reduce the number of pixels you're working on.

Thanks Rob. But is the "uncompressed data" you are referring to a interim variable that is used for processing the JPEG? why is this memory retained after the UIImage.jpegData()? — user14230441, Sep 07 '20 at 00:40
In order to display the image (i.e. in an image view), you need an uncompressed copy. But my other point was that you haven't given us a lot of code, and it's possible that you're storing the result of `generateImage` in a property, in which case it wouldn't be released. — Rob Napier, Sep 07 '20 at 02:00

user14230441 · Answer 2 · 2020-09-07T09:05:52.337

-2

Interestingly, the following code resolves the issue, and apparently the issue comes from the generateImage() method but I cannot explain why.

func generateImage() -> Data {
    // NOTE do not put any of these codes out in case of retain issues of
    // the JPEG data or the images.
    let canvasSize = self.outputImage.size
    UIGraphicsBeginImageContext(canvasSize)
    self.imageView.zoomView?.layer.render(in: UIGraphicsGetCurrentContext()!)
    let newImage = UIGraphicsGetImageFromCurrentImageContext()?.jpegData(compressionQuality: 0)
    UIGraphicsEndImageContext()
    return newImage!
}

I updated this method to return the data directly rather than return a UIImage, and fortunately with this update, the memory for converting the UIImage to JPEG data as Rob mentioned is not retained after the calls to put the Data into some class variables/caches. But I still do not quite understand the reason though. Probably people can comment more?

edited Sep 07 '20 at 09:05

answered Sep 07 '20 at 00:43

user14230441

1
1

Is this the answer? Or are you asking an extension to the question? And what is `generateMosaicImage`? – matt Sep 07 '20 at 00:46
This is going to release the uncompressed data before it returns rather than returning the uncompressed data. Depending on what you do with the returned value, this could make perfect sense. It's very hard to tell because you don't show most of the important code, and the code you post doesn't quite match the code you describe, so we're guessing a lot. For example, when you changed `generateImage()`, you clearly also changed the calling code because the signature is different. But you haven't shown us that change. – Rob Napier Sep 07 '20 at 02:02

Why in Swift5, the UIImage.jpegData() call will retain that much memory?

2 Answers2