CameraX implementation with ZXing 1D barcode reader, ImageProxy is rotated 90 degrees from the target and can't find a way to rotate it

Question

I am trying to use CameraX with ZXing library to identify barcodes, I am using ImageAnalyzer to get ImageProxy and feed its byte array to the PlanarYUVLuminanceSource to handle it by ZXing. My target rotation is 0 and the imageProxy that is coming from the camera is 90 degrees, so the barcode is not readable until I rotate it 90 degrees. here is the code:

private void buildAnalyzer() {
    if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.LOLLIPOP) {
        imageAnalysis = new ImageAnalysis.Builder()
                .setTargetResolution(new Size(1080, 720))
                .setTargetRotation(Surface.ROTATION_0)
                .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                .build();
    }

    MultiFormatReader reader = new MultiFormatReader();

    Hashtable<DecodeHintType, Object> hints = new Hashtable<DecodeHintType, Object>(
            2);
    Vector<BarcodeFormat> decodeFormats = new Vector<BarcodeFormat>();

    decodeFormats = new Vector<BarcodeFormat>();

    decodeFormats.add(BarcodeFormat.EAN_13);

    hints.put(DecodeHintType.POSSIBLE_FORMATS, decodeFormats);
    hints.put(DecodeHintType.TRY_HARDER, Boolean.TRUE);

    reader.setHints(hints);


    imageAnalysis.setAnalyzer(Executors.newFixedThreadPool(1), new ImageAnalysis.Analyzer() {
        @Override
        public void analyze(@NonNull ImageProxy imageProxy) {

            int rotationDegrees = imageProxy.getImageInfo().getRotationDegrees();
            Log.v(TAG, "Rotation + " + rotationDegrees);
            Log.v(TAG, "format + " + imageProxy.getFormat());


            
           
            ImageProxy.PlaneProxy[] planes = imageProxy.getPlanes();
            Log.v(TAG, "planes " + planes.length);

            ByteBuffer yBuffer = planes[0].getBuffer();
            ByteBuffer uBuffer = planes[1].getBuffer();
            ByteBuffer vBuffer = planes[2].getBuffer();

            int ySize = yBuffer.remaining();
            int uSize = uBuffer.remaining();
            int vSize = vBuffer.remaining();

            byte[] data = new byte[ySize + uSize + vSize];
            //U and V are swapped
            yBuffer.get(data, 0, ySize);
            vBuffer.get(data, ySize, vSize);
            uBuffer.get(data, ySize + vSize, uSize);

            Log.v(TAG, "width : " + imageProxy.getWidth());
            Log.v(TAG, "height : " + imageProxy.getHeight());


           
            Log.v(TAG, "planes 1 size : " + ySize + " remaining: " + yBuffer.remaining());
            Log.v(TAG, "planes 2 size : " + uSize + " remaining: " + uBuffer.remaining());
            Log.v(TAG, "planes 3 size : " + vSize + " remaining: " + vBuffer.remaining());


            Log.v(TAG, "data length : " + data.length);


            PlanarYUVLuminanceSource source = new PlanarYUVLuminanceSource(data,
                    imageProxy.getWidth(),
                    imageProxy.getHeight()
                    , 0, 0,
                    imageProxy.getWidth(),
                    imageProxy.getHeight()
                    , false);
            BinaryBitmap binary = new BinaryBitmap(new HybridBinarizer(source));


            try {
                Result result = reader.decodeWithState(binary);
                Log.d(TAG, "reader read " + result.getText());
                Looper looper = Looper.getMainLooper();
                Handler handler = new Handler(looper);
                handler.post(new Runnable() {
                    @Override
                    public void run() {
                        Toast.makeText(con, result.getText() + "\n" + result.getNumBits(), Toast.LENGTH_LONG).show();

                    }
                });

            } catch (NotFoundException e) {
                Log.d(TAG, "exception  " + e.getMessage());

            }



            imageProxy.close();
        }
    });
}

The code above works fine which indicates that there is nothing wrong with the byte, that being said, here is the solutions that I tried and never worked:

1- rotate the byte array: I used this method https://stackoverflow.com/a/15775173/4674191 to rotate the byte array "data" that I extracted from the ImageProxy but it returned an error array out of bound exception, this method works on arrays with factor of 4 and my byte arrays doesn't meet this condition. logically the byte array is a yuv420 image format , consist of 3 layers with certain width and height. but what drives me crazy that the width and the height are not compatible with the array :

width = 1440 , height = 1080 , planes 1 size : 1589728 , planes 2 size : 794847 , planes 3 size : 794847

looking at the numbers they are not dividable by 1440 nor 1080 , and not a factor of 4.

2- create a bitmap from the byte array and rotate it, then extract the rotated byte array from the new bitmap: also doesn't work because the width and height is unknown just like earlier the resolution is not compatible with the byte array numbers and the result bitmap is bunch of distorted green mess.

3- the activity is locked on portrait mode from the manifest android:screenOrientation="portrait" : same problem, imageProxy rotation is still 90

4- PlanarYUVLuminanceSource doesnt support rotateion :( don't know why.

I tried almost all the solutions on StackOverFlow and read all the documentations for cameraX and nothing seems to address this problem. if there is a way to know the actual resolution for the byteArray it would solve the problem I guess.

Answer 1

byte[] data = ImageUtil.imageToJpegByteArray(imageProxy);
Bitmap bitmap= BitmapFactory.decodeByteArray(data, 0, data.size)

you can get bitmap by code above. ImageUtil class is in androidx.camera.core

androidx.camera.core.internal.utils.ImageUtil

Answer 2

I have implemented this extension function after a lot of research:

fun ImageProxy.toBitmap(): Bitmap {
    val buffer = planes[0].buffer.apply { rewind() }
    val bytes = ByteArray(buffer.capacity())

    // Get bitmap
    buffer.get(bytes)
    val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)

    // Fix rotation if needed
    val angle = imageInfo.rotationDegrees.toFloat()
    val matrix = Matrix().apply { postRotate(angle) }

    // Return rotated bitmap
    return Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
}

You can get the ImageProxy by calling takePicture from the android camerax library:

imageCapture.takePicture(cameraExecutor, object : ImageCapture.OnImageCapturedCallback() {
    override fun onCaptureSuccess(imageProxy: ImageProxy) {
        val bitmap = imageProxy.toBitmap()
        imageProxy.close()
    }
})