Fresco custom image decoder resize options are not applied

Question

I have a custom decoder that uses Conceal to load encrypted images from local storage.

Everything works (images are displayed) but performance is terrible when loading local camera images since no downsampling or bitmap resizing is applied at all when the actual JPEG decoder does it work.

class CryptoImageDecoder(val crypto: Crypto, val poolFactory: PoolFactory) : ImageDecoder {

    val defaultDecoder = DefaultImageDecoder(null,
            Fresco.getImagePipelineFactory().platformDecoder,
            Bitmap.Config.ARGB_8888)

    override fun decode(encodedImage: EncodedImage,
                        length: Int,
                        qualityInfo: QualityInfo?,
                        options: ImageDecodeOptions?): CloseableImage {

        encodedImage.use {
            val inputStream = encodedImage.inputStream
            inputStream.skip(CRYPTO_HEADER_BYTES.size.toLong()) //Skip header
            val cipherInputStream = crypto.getCipherInputStream(inputStream, CRYPTO_ENTITY)

            cipherInputStream.use {
                val bytes = poolFactory.pooledByteBufferFactory.newByteBuffer(cipherInputStream)
                val bytesClosable = CloseableReference.of(bytes)
                val clearEncodedImage = EncodedImage(bytesClosable)

                //This is always 1, no matter what resizeOptions I use in the request
                //clearEncodedImage.sampleSize = how to calculate this?
                clearEncodedImage.encodedCacheKey = encodedImage.encodedCacheKey

                return defaultDecoder.decode(clearEncodedImage, bytes.size(), qualityInfo, options)
            }
        }
    }
}

The way the request is done pretty straightforward

val request = ImageRequestBuilder.newBuilderWithSource(attachment.sourceImageUri)
        .setSource(attachment.sourceImageUri)
        .setResizeOptions(ResizeOptions.forSquareSize(300))
        .build()

val controller = Fresco.newDraweeControllerBuilder()
        .setOldController(holder.draweeView.controller)
        .setImageRequest(request)
        .build()

Why are resize options ignored, is there another option that I am missing for the decoder?

How did you get the instance of PoolFactory here? I am trying something similar but I am not sure how to get the poolfactory. — vineeth, Apr 13 '18 at 18:13

score 1 · Answer 1 · answered Jun 14 '17 at 16:12

1

I believe that resizing is done before decoding (i.e. the image is transcoded to a smaller JPEG) - and only JPEG is supported right now.

Take a look at ResizeAndRotateProducer.

answered Jun 14 '17 at 16:12

Alexander Oprisnik

1,212
9
9

Interesting, that seems to be the piece I am missing. Any idea about how to integrate that producer for my decoder? Doesn't look like there is a straightforward way to get a ProducerContext to extract the request from my decoder. – minivac Jun 16 '17 at 11:31
I think the easiest way for this to work is to supply a custom decoder that decrypts and decodes. We don't have a lot of documentation for this yet, since this is still changing a bit, but you can take a look at the sample app for SVG / Keyframes support. You add a new image format, header checker and then a decrypting decoder that internally forwards to the default decoder once decoded. – Alexander Oprisnik Jun 16 '17 at 12:41

score 0 · Answer 2 · answered Jun 16 '17 at 15:28

Following the example relevant implemetation given by Alexander Oprisnik I got resizing working with my custom decoder, these are the relevant parts of the solution:

class CryptoImageDecoder(val crypto: Crypto, val poolFactory: PoolFactory) : ImageDecoder {

    val defaultDecoder by lazy {
        DefaultImageDecoder(null,
                Fresco.getImagePipelineFactory().platformDecoder,
                Bitmap.Config.ARGB_8888)
    }

    override fun decode(encodedImage: EncodedImage,
                        length: Int,
                        qualityInfo: QualityInfo?,
                        options: ImageDecodeOptions?): CloseableImage {

        var cipherInputStream: InputStream? = null
        var clearEncodedImage: EncodedImage? = null
        var transcodedImage: EncodedImage? = null
        var transcodedRef: CloseableReference<PooledByteBuffer>? = null

        try {
            val inputStream = encodedImage.inputStream
            inputStream.skip(CRYPTO_HEADER_BYTES.size.toLong()) //Skip header
            cipherInputStream = crypto.getCipherInputStream(inputStream, CRYPTO_ENTITY)

            val bytes = poolFactory.pooledByteBufferFactory.newByteBuffer(cipherInputStream)
            val bytesClosable = CloseableReference.of(bytes)
            clearEncodedImage = EncodedImage(bytesClosable)

            val dimensions = BitmapUtil.decodeDimensions(clearEncodedImage.inputStream)
            clearEncodedImage.width = dimensions?.first ?: -1
            clearEncodedImage.height = dimensions?.second ?: -1
            clearEncodedImage.rotationAngle = 0

            val decodeOptions = options as? CryptoDecodeOptions ?: error("ImageOptions should be CryptoDecodeOptions")
            val imageRequest = decodeOptions.imageRequest
            val downsampleRatio = DownsampleUtil.determineSampleSize(imageRequest, clearEncodedImage)
            val downsampleNumerator = calculateDownsampleNumerator(downsampleRatio)

            if (downsampleNumerator == JpegTranscoder.SCALE_DENOMINATOR) {
                //No need to apply any transformation
                return defaultDecoder.decode(clearEncodedImage, bytes.size(), qualityInfo, options)
            }

            val outputStream = poolFactory.pooledByteBufferFactory.newOutputStream()

            JpegTranscoder.transcodeJpeg(
                    PooledByteBufferInputStream(bytes),
                    outputStream,
                    0, //Rotation is ignored
                    downsampleNumerator,
                    DEFAULT_JPEG_QUALITY)
            val bb = outputStream.toByteBuffer()
            transcodedRef = CloseableReference.of(bb)
            transcodedImage = EncodedImage(transcodedRef)
            transcodedImage.encodedCacheKey = encodedImage.encodedCacheKey
            return defaultDecoder.decode(transcodedImage, bb.size(), qualityInfo, options)
        } catch (ex: Exception) {
            Grove.e { "Something went wrong decoding the image" }
            throw ex
        } finally {
            cipherInputStream?.close()
            clearEncodedImage?.close()
            transcodedImage?.close()
            transcodedRef?.close()
        }
    }

    private fun calculateDownsampleNumerator(downsampleRatio: Int): Int {
        return Math.max(1, JpegTranscoder.SCALE_DENOMINATOR / downsampleRatio)
    }
}

/**
 * Dummy wrapper that hold a reference to the request that used this options, required
 * to perform jpeg resizing
 */
class CryptoDecodeOptions(builder: ImageDecodeOptionsBuilder) : ImageDecodeOptions(builder) {
    internal lateinit var imageRequest: ImageRequest
}

/**
 * Decoded images need the actual request to determine resize operations since
 * transcoding is not possible with encrypted images.
 */
fun ImageRequestBuilder.buildForCrypto(): ImageRequest {
    val cryptoDecodeOptions = CryptoDecodeOptions(ImageDecodeOptionsBuilder())
    this.imageDecodeOptions = cryptoDecodeOptions
    val request = this.build()
    cryptoDecodeOptions.imageRequest = request
    return request
}

The trick is to use CryptoDecodeOptions that simply holds a reference to the request that holds the resize parameters. The rest of the code is a simplification of the method doTransform from ResizeAndRotateProducer. Rotation is ignored since my application does not require it.

Fresco custom image decoder resize options are not applied

2 Answers2